Question

3. Using k-NN method, explain an algorithm in as much detail as possible for how to conduct prediction and classification. Please also describe how to evaluate the performance of prediction and classification. In your description, please provide all the notations and formulas

Answer 1

K-Nearest Neighbour (KNN) as Classifier:-

KNN is a simple algorithm that stores all available cases and classifies new cases based on a similarity measures. Nearest neighbours have been used in statistical estimation and pattern recognization already in the begining of 1970's (non-parametric technique).

Let training data set has X1, X2, ......., Xn instance and each instance belongs to either C1 and C2 class. Our task is to determine class for new instance say Xnew.

1) We deside k = no. of nearest neighbour.

2) If k=1, then find the closest point Xi to Xnew using any distance measure and classify Xnew to class of Xi (either C1 or C2).

3) If k>1 then find k closest training points to Xnew by using any distance meaures and classify to mejority vot among the K-points.