Method for (RefLink[Classify, paclet:ref/Classify]) and (RefLink[Predict, paclet:ref/Predict]).

Infers the class or value of a new example by analyzing its nearest neighbors in the feature space.

Nearest neighbors is a type of instance-based learning. In its simplest form, it picks the commonest class or averages the values among the k nearest neighbors.

The following options can be given:

Possible settings for "NearestMethod" include:

OpenersPrimaryExamplesSection(Spacer1)Examples  (8)

OpenersExampleSection0(Spacer1)Basic Examples  (2)

Train a classifier function on labeled examples:

c=Classify[{1,2,3,4}->{1,1,2,2},Method->"NearestNeighbors"]

Obtain information about the classifier:

Information[c]

Classify a new example:

c[1.3]

Generate some data and visualize it:

data=Table[x->x+RandomVariate[NormalDistribution[0, 2]], {x, RandomReal[{-10, 10}, 40]}];
ListPlot[List@@@data]

Train a predictor function on it:

p=Predict[data,Method->"NearestNeighbors"]

Compare the data with the predicted values and look at the standard deviation:

Show[Plot[{p[x],
p[x]+StandardDeviation[p[x,"Distribution"]],p[x]-StandardDeviation[p[x,"Distribution"]]},
{x,-2,6},
PlotStyle->{Blue,Gray,Gray},
Filling->{2->{3}},
Exclusions->False,
PerformanceGoal->"Speed",PlotLegends->{"Prediction","Confidence Interval"}],ListPlot[List@@@data,PlotStyle->Red,PlotLegends->{"Data"}]]

OpenersExampleSection1(Spacer1)Options  (6)

OpenersExampleSubsection0(Spacer1)"NeighborsNumber"  (2)

Train a predictor function using a specific "NeighborsNumber":

Predict[{1.98`,3.83`,1.69`,0.04`,2.48`,1.66`}->{-1.41,-0.71,-0.701,-0.4,-1.91,-1.6}, Method->{"NearestNeighbors","NeighborsNumber"->2}]

Generate a labeled training set and visualize it:

trainigset=Table[x->Sin[4x]+RandomReal[.4], {x,RandomReal[{0, 6},30]}];
ListPlot[List@@@trainigset]

Train a predictor using a small "NeighborsNumber":

p2=Predict[trainigset, Method->{"NearestNeighbors","NeighborsNumber"->2}];

Train a predictor using a large "NeighborsNumber":

p10=Predict[trainigset, Method->{"NearestNeighbors","NeighborsNumber"->10}];

Compare the two predictors:

Plot[{p2[x], p10[x]}, {x, 0, 6}]

OpenersExampleSubsection1(Spacer1)"DistributionSmoothing"  (2)

Train a classifier using the "DistributionSmoothing" suboption:

Classify[{1.98`,3.83`,1.69`,0.04`,2.48`,1.66`}->{"a","a","b","b","a","b"}, Method->{"NearestNeighbors","DistributionSmoothing"->2}]

Train two classifiers on an imbalanced dataset by varying the value of "DistributionSmoothing":

data={1->True,2->True,3->True,4->True,5->False,6->True};

c1=Classify[data, Method->{"NearestNeighbors","DistributionSmoothing"->.1}]

c2=Classify[data, Method->{"NearestNeighbors","DistributionSmoothing"->10}]

Look at the probabilities for the two classifiers:

c1[5,"Probabilities"]

c2[5,"Probabilities"]

OpenersExampleSubsection2(Spacer1)"NearestMethod"  (2)

Train a classifier using a specific "NearestMethod":

Classify[{1.98`,3.83`,1.69`,0.04`,2.48`,1.66`}->{"a","a","b", "b","a","b"}, Method->{"NearestNeighbors","NearestMethod"->"Scan"}]

Generate a large dataset and visualize it:

gaussian[μ_,σ_,n_]:=RandomVariate[MultinormalDistribution[μ,{{σ,0},{0,σ}}],n];
positions={{4,2},{-2,2},{0,-3},{3,0}};
sizes={2,1,5,0.5};
colors={, , ,};
nums={10000,10000,50000,20000};

clusters=MapThread[gaussian,{positions,sizes,nums}];
trainigset=AssociationThread[colors,clusters];
plot=ListPlot[clusters,PlotStyle->Darker[colors,0.1],ImageSize->200,PlotRange->{{-5,5},{-5,5}},Frame->True,AspectRatio->1,PlotLabel->"data"]

Train several classifiers using the different methods and compare their training times:

classifiers=Classify[trainigset, Method->{"NearestNeighbors","NearestMethod"->#}]&/@{"Octree","KDtree","Scan"};

Compare the corresponding training times:

ClassifierInformation[#,"TrainingTime"]&/@classifiers

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[Classify, paclet:ref/Classify])  ▪  (RefLink[Predict, paclet:ref/Predict])  ▪  (RefLink[ClassifierFunction, paclet:ref/ClassifierFunction])  ▪  (RefLink[PredictorFunction, paclet:ref/PredictorFunction])  ▪  (RefLink[ClassifierMeasurements, paclet:ref/ClassifierMeasurements])  ▪  (RefLink[PredictorMeasurements, paclet:ref/PredictorMeasurements])  ▪  (RefLink[SequencePredict, paclet:ref/SequencePredict])  ▪  (RefLink[ClusterClassify, paclet:ref/ClusterClassify])  ▪    ▪    ▪    ▪    ▪    ▪    ▪    ▪    ▪    ▪  }, {Related Machine Learning Methods:  (RefLink[DecisionTree, paclet:ref/method/DecisionTree])  ▪  (RefLink[LinearRegression, paclet:ref/method/LinearRegression])  ▪  (RefLink[LogisticRegression, paclet:ref/method/LogisticRegression])  ▪  (RefLink[GaussianProcess, paclet:ref/method/GaussianProcess])  ▪  (RefLink[GradientBoostedTrees, paclet:ref/method/GradientBoostedTrees])  ▪  (RefLink[Markov, paclet:ref/method/Markov])  ▪  (RefLink[NaiveBayes, paclet:ref/method/NaiveBayes])  ▪  (RefLink[NeuralNetwork, paclet:ref/method/NeuralNetwork])  ▪  (RefLink[RandomForest, paclet:ref/method/RandomForest])  ▪  (RefLink[SupportVectorMachine, paclet:ref/method/SupportVectorMachine])}})}})

({{(Spacer1), ({{(Spacer1)Related Links}, {(WebLink[DocumentationFooterIconsRelatedLinks-EIWL.png(Spacer1)An Elementary Introduction to the Wolfram Language, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])(WebLink[: Machine Learning, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])}})}})