IBK Optimal K

[Mike1961]

Hi Mark,

I am trying to verify the K value found in IBK(crossvalidate = True and KNN set to max (number of instances) is correct.

When using the crossvalidate IBK, I use that on "Use training set" under Classify. Say that IBK optimize KNN gives me a correlation of .5 and optimal K of 10 NN.

Then I try to verify by switching crossvalidate to false and setting KNN to 10. Then I run a LOOCV under "Cross-validation" but get wildly different results from the correlation value found by optimizing on "Use training set".



On a similar experiment using CVparmSelection with IBK. I set K 1 100 100. There are 100 instances. I do the CVparmSelection on "Use training set" with number of folds set to number of instances so it is a LOOCV,

But when I get the K value found by CVParmSelection and then just run plain IBK with that value on "Cross-validation" with a LOOCV, the correlations between the finding the optimum by CVPS and using that optimum are again way different.


What am I misunderstanding?


Thanks,
Mike

[Mike1961]

Mark,

Never mind on that, I figured it out. But, it would be nice to have a toggle on the IBK KNN optimize to have it output the CV'd value too.