The k nearest neighbors method (abbreviated k-NN) is a supervised classification method (based on a training set and test set) that serves to estimate the density function F(x / Cj) of the predictors x for each class Cj.

This is a non-parametric classification method, which estimates the value of the probability density function or directly the a posteriori probability that an element x belongs to the class Cj from the information provided by the set of observations. In the learning process, no assumption is made about the distribution of the predictor variables.

In the recognition of patterns, the algorithm k-NN is used as a method of classifying objects (elements) based on a training using close examples in the space of the elements. k-NN is a type of lazy learning, where the function is approached locally.

The training examples are points of a multidimensional space. Such a point is assigned to a class C if it is the most frequent class among the k nearest training examples. Generally the Euclidean distance is used.

USE CASE : BREAST CANCER DIAGNOSIS

This use case shows how to use the K Nearest Neighbors method to classify whether a patient will survive or die within the next 5 years after a breast cancer has been diagnosed. For each patient, the diagnosis is done in terms of number of axillary nodes which are lymph nodes that become significant in case of breast cancer. Each rows provides the age and the number of axillary nodes of the patient. Here follows the first 40 rows over 306 of the dataset :

Dataset of patients that survived or died within 5 years after a breast cancer diagnosis according to his age and the number of axillary nodes.

Columns :
– AGE : the age of the patient
– AXILLARY_NODES : the number of axillary nodes observed on the patient
– DEATH_WITHIN_5_YEARS : ‘1’ if the patient died within 5 years after the breast cancer diagnosis, and ‘0’ if he survived

Here follows the complete dataset in CSV format :

AGE,AXILLARY_NODES,DEATH_WITHIN_5_YEARS
30,1,0
30,3,0
30,0,0
31,2,0
31,4,0
33,10,0
33,0,0
34,0,1
34,9,1
34,30,0
34,1,0
34,10,0
34,7,0
34,0,0
35,13,0
35,0,0
36,1,0
36,0,0
37,0,0
37,0,0
37,0,0
37,6,0
37,15,0
37,0,0
38,21,1
38,2,0
38,0,0
38,0,0
38,3,0
38,1,0
38,0,0
38,11,0
38,1,0
38,5,0
39,0,1
39,0,0
39,0,0
39,0,0
39,2,0
39,4,0
40,2,0
40,0,0
40,0,0
41,23,1
41,0,1
41,0,1
41,0,0
41,8,0
41,0,0
41,0,0
41,8,0
41,0,0
41,0,0
42,1,1
42,0,1
42,0,0
42,1,0
42,2,0
42,4,0
42,20,0
42,0,0
42,1,0
43,52,1
43,2,1
43,0,1
43,0,1
43,14,0
43,2,0
43,3,0
43,0,0
43,2,0
43,0,0
43,4,0
44,6,1
44,9,1
44,19,1
44,0,0
44,1,0
44,0,0
44,16,0
45,6,1
45,0,1
45,1,1
45,0,0
45,0,0
45,14,0
45,0,0
45,0,0
45,1,0
46,2,1
46,3,1
46,5,1
46,20,1
46,0,0
46,3,0
46,0,0
47,23,1
47,0,1
47,0,1
47,0,0
47,6,0
47,0,0
47,0,0
47,3,0
47,4,0
47,4,0
47,12,0
48,11,1
48,11,1
48,7,1
48,8,0
48,2,0
48,0,0
48,0,0
49,0,1
49,10,1
49,1,0
49,0,0
49,0,0
49,1,0
49,1,0
49,3,0
49,0,0
49,1,0
50,13,1
50,0,1
50,0,0
50,6,0
50,0,0
50,1,0
50,1,0
50,2,0
50,0,0
50,0,0
50,4,0
50,1,0
51,13,1
51,3,1
51,7,0
51,1,0
51,0,0
51,1,0
52,3,1
52,2,1
52,3,1
52,4,1
52,0,0
52,4,0
52,0,0
52,4,0
52,5,0
52,0,0
52,1,0
52,0,0
52,0,0
52,0,0
53,4,1
53,1,1
53,3,1
53,9,1
53,24,1
53,12,1
53,1,0
53,1,0
53,2,0
53,1,0
53,0,0
54,11,1
54,23,1
54,5,1
54,7,1
54,7,0
54,3,0
54,0,0
54,46,0
54,0,0
54,7,0
54,19,0
54,1,0
54,0,0
55,6,1
55,15,1
55,1,0
55,0,0
55,1,0
55,18,0
55,0,0
55,3,0
55,22,0
55,1,0
56,9,1
56,3,1
56,0,0
56,2,0
56,1,0
56,0,0
56,0,0
57,5,1
57,14,1
57,1,1
57,9,0
57,0,0
57,0,0
57,0,0
57,0,0
57,0,0
57,0,0
57,0,0
58,0,0
58,3,0
58,1,0
58,0,0
58,0,0
58,3,0
58,2,0
59,35,1
59,0,0
59,0,0
59,1,0
59,4,0
59,0,0
59,7,0
59,3,0
60,17,1
60,0,1
60,1,0
60,2,0
60,25,0
60,0,0
61,5,1
61,0,1
61,1,1
61,0,0
61,0,0
61,0,0
61,8,0
61,0,0
61,0,0
62,13,1
62,0,1
62,19,1
62,6,0
62,0,0
62,0,0
62,0,0
63,1,1
63,0,0
63,0,0
63,0,0
63,0,0
63,0,0
63,9,0
63,28,0
64,0,0
64,22,0
64,0,0
64,0,0
64,0,0
65,0,1
65,2,1
65,22,1
65,15,1
65,0,0
65,0,0
65,0,0
65,2,0
65,0,0
65,1,0
66,0,1
66,13,1
66,0,0
66,1,0
66,0,0
67,8,1
67,1,1
67,0,0
67,0,0
67,0,0
67,0,0
68,0,0
68,0,0
69,8,1
69,0,0
69,0,0
69,0,0
70,0,1
70,4,1
70,14,0
70,0,0
70,0,0
70,8,0
70,0,0
71,2,0
72,0,1
72,0,0
72,0,0
72,3,0
73,0,0
73,0,0
74,3,1
74,0,0
75,1,0
76,0,0
77,3,0
78,1,1
83,2,1