Method for (RefLink[LearnDistribution, paclet:ref/LearnDistribution]).

Models probability density with a mixture of simple distributions.

"KernelDensityEstimation" is a nonparametric method that models the probability density of a numeric space with a mixture of simple distributions (called kernels) centered around each training example, as in (RefLink[KernelMixtureDistribution, paclet:ref/KernelMixtureDistribution]).

The probability density function for a vector x is given by (1/(m h))(∑_(i=1)^m)k(((x-(Subscript[x, i]))/h)) for a kernel function k, kernel size h and a number of training examples m.

The following options can be given:

Possible settings for "KernelType" include:

Possible settings for (RefLink[Method, paclet:ref/Method]) include:

When "KernelType"->"Gaussian", each kernel is a spherical Gaussian (product of independent normal distributions (1/(Sqrt[2π]))(e^(-((u^2)/(2(h^2))))) ), and "KernelSize" h refers to the standard deviation of the normal distribution.

When "KernelType"->"Ball", each kernel is a uniform distribution inside a sphere, and "KernelSize" refers to the radius of the sphere.

The value of "NeighborsNumber"->k is converted into kernel size(s), so that a kernel centered around a training example typically "contains" k other training examples. If "KernelType"->"Ball", "contains" refers to examples that are inside the ball. If "KernelType"->"Gaussian", "contains" refers to examples that are inside a ball of radius h(Sqrt[n]) where n is the dimension of the data.

When (RefLink[Method, paclet:ref/Method])->"Fixed" and "NeighborsNumber"->k, a unique kernel size is found such that training examples contain on average k other examples.

When (RefLink[Method, paclet:ref/Method])->"Adaptive" and "NeighborsNumber"->k, each training example adapts its kernel size such that it contains about k other examples.

Because of preprocessing, the "NeighborsNumber" option is typically a more convenient way to control kernel sizes than "KernelSize". When (RefLink[Method, paclet:ref/Method])->"Fixed", the value of "KernelSize" supersedes the value of "NeighborsNumber".

(RefLink[Information, paclet:ref/Information])[(RefLink[LearnedDistribution, paclet:ref/LearnedDistribution])[…],"MethodOption"] can be used to extract the values of options chosen by the automation system.

(RefLink[LearnDistribution, paclet:ref/LearnDistribution])[…,(RefLink[FeatureExtractor, paclet:ref/FeatureExtractor])->"Minimal"] can be used to remove most preprocessing and directly access the method.

OpenersPrimaryExamplesSection(Spacer1)Examples  (7)

OpenersExampleSection0(Spacer1)Basic Examples  (3)

Train a "KernelDensityEstimation" distribution on a numeric dataset:

ld=LearnDistribution[{1.2,2.1,3.5,4.3},Method->"KernelDensityEstimation"]

Look at the distribution (RefLink[Information, paclet:ref/Information]):

Information[ld]

Obtain options information:

Information[ld,"MethodOption"]

Obtain an option value directly:

Information[ld,"KernelType"]

Compute the probability density for a new example:

PDF[ld,1.3]

Plot the PDF along with the training data:

Show[Plot[PDF[ld,x],{x,-6,12},Filling->Bottom],NumberLinePlot[{1.2,2.1,3.5,4.3},Spacings->0, PlotStyle->Red]]

Generate and visualize new samples:

Histogram[RandomVariate[ld,10000],50]

Train a "KernelDensityEstimation" distribution on a two-dimensional dataset:

iris=ExampleData[{"MachineLearning","FisherIris"},"Data"][[All,1,{1,3}]];

ld=LearnDistribution[iris,Method->"KernelDensityEstimation"]

Plot the PDF along with the training data:

Show[ContourPlot[PDF[ld,{x,y}],{x,4,8},{y,1,7},PlotRange->All,Contours->10],ListPlot[iris, PlotStyle->Red]]

Use (RefLink[SynthesizeMissingValues, paclet:ref/SynthesizeMissingValues]) to impute missing values using the learned distribution:

SynthesizeMissingValues[ld,{5.5,Missing[]}]

Histogram[Table[Last@SynthesizeMissingValues[ld,{5.5,Missing[]}],1000]]

Train a "KernelDensityEstimation" distribution on a nominal dataset:

ld=LearnDistribution[{"A","A","B","B","B"},Method->"KernelDensityEstimation"]

Because of the necessary preprocessing, the PDF computation is not exact:

PDF[ld,"A"]

PDF[ld,"A"]

Use (RefLink[ComputeUncertainty, paclet:ref/ComputeUncertainty]) to obtain the uncertainty on the result:

PDF[ld,"A",ComputeUncertainty->True]

Increase (RefLink[MaxIterations, paclet:ref/MaxIterations]) to improve the estimation precision:

PDF[ld,"A",MaxIterations->1000,ComputeUncertainty->True]

OpenersExampleSection1(Spacer1)Options  (4)

OpenersExampleSubsection0(Spacer1)"KernelType"  (1)

Train a "KernelDensityEstimation" distribution with a "Ball" kernel:

iris=ExampleData[{"MachineLearning","FisherIris"},"Data"][[All,1,{1,3}]];

ld=LearnDistribution[iris,Method->{"KernelDensityEstimation","KernelType"->"Ball"}]

Evaluate the PDF of the distribution at a specific point:

PDF[ld,{6,4}]

Visualize the PDF obtained after training a kernel mixture distribution with a "Ball" and a "Gaussian" kernel:

plots=(ld=LearnDistribution[iris,Method->{"KernelDensityEstimation","KernelType"->#},FeatureExtractor->"Minimal"];
Show[ContourPlot[PDF[ld,{x,y}],{x,2,10},{y,0,8},PlotRange->All,Contours->10,PlotLabel->#],ListPlot[iris, PlotStyle->Red],ImageSize->250]) & /@{"Ball","Gaussian"};

Row[plots]

OpenersExampleSubsection1(Spacer1)"NeighborsNumber"  (1)

Train a kernel mixture distribution with a kernel size of about 10 neighbors:

iris=ExampleData[{"MachineLearning","FisherIris"},"Data"][[All,1,{1,3}]];

ld=LearnDistribution[iris,Method->{"KernelDensityEstimation","NeighborsNumber"->10}]

Evaluate the PDF of the distribution at a specific point:

PDF[ld,{6,4}]

Visualize the PDF obtained after training a kernel mixture distribution with various kernel sizes expressed as neighbors numbers:

plots=(
ld=LearnDistribution[iris,Method->{"KernelDensityEstimation","NeighborsNumber"->#},FeatureExtractor->"Minimal"];
Show[ContourPlot[PDF[ld,{x,y}],{x,2,10},{y,0,8},PlotRange->All,Contours->10],ListPlot[iris, PlotStyle->Red],ImageSize->250,PlotLabel->"k = "<>ToString[#]]
) & /@ {1,2,5,10,20,50};

Grid[Partition[plots,2]]

OpenersExampleSubsection2(Spacer1)"KernelSize"  (1)

Train a kernel mixture distribution with a kernel size of 0.2:

iris=ExampleData[{"MachineLearning","FisherIris"},"Data"][[All,1,{1,3}]];

ld=LearnDistribution[iris,Method->{"KernelDensityEstimation","KernelSize"->0.2}]

Evaluate the PDF of the distribution at a specific point:

PDF[ld,{6,4}]

Visualize the PDF obtained after training a kernel mixture distribution with various kernel sizes:

plots=(
ld=LearnDistribution[iris,Method->{"KernelDensityEstimation","KernelSize"->#},FeatureExtractor->"Minimal"];
Show[ContourPlot[PDF[ld,{x,y}],{x,2,10},{y,0,8},PlotRange->All,Contours->10],ListPlot[iris, PlotStyle->Red],ImageSize->250,PlotLabel->"h = "<>ToString[#]]
) & /@ {0.05,0.1,0.2,0.35,0.5,1};

Grid[Partition[plots,2]]

OpenersExampleSubsection3(Spacer1)Method  (1)

Train a "KernelDensityEstimation" distribution with the "Adaptive" method:

iris=ExampleData[{"MachineLearning","FisherIris"},"Data"][[All,1,{1,3}]];

ld=LearnDistribution[iris,Method->{"KernelDensityEstimation",Method->"Adaptive"}]

Evaluate the PDF of the distribution at a specific point:

PDF[ld,{6,4}]

Visualize the PDF obtained after training a kernel mixture distribution with a "Ball" and a "Gaussian" kernel:

plots=(ld=LearnDistribution[iris,Method->{"KernelDensityEstimation",Method->#},FeatureExtractor->"Minimal"];
Show[ContourPlot[PDF[ld,{x,y}],{x,2,10},{y,0,8},PlotRange->All,Contours->10,PlotLabel->#],ListPlot[iris, PlotStyle->Red],ImageSize->250]) & /@ {"Fixed","Adaptive"};

Row[plots]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[LearnDistribution, paclet:ref/LearnDistribution])  ▪  (RefLink[LearnedDistribution, paclet:ref/LearnedDistribution])  ▪  (RefLink[KernelMixtureDistribution, paclet:ref/KernelMixtureDistribution])  ▪  (RefLink[MixtureDistribution, paclet:ref/MixtureDistribution])  ▪  (RefLink[MultinormalDistribution, paclet:ref/MultinormalDistribution])  ▪  (RefLink[Nearest, paclet:ref/Nearest])  ▪  (RefLink[AnomalyDetection, paclet:ref/AnomalyDetection])  ▪  (RefLink[RandomVariate, paclet:ref/RandomVariate])  ▪  (RefLink[PDF, paclet:ref/PDF])  ▪  (RefLink[RarerProbability, paclet:ref/RarerProbability])  ▪    ▪    ▪    ▪  }, {Related Machine Learning Methods:  (RefLink[GaussianMixture, paclet:ref/method/GaussianMixture])  ▪  (RefLink[ContingencyTable, paclet:ref/method/ContingencyTable])  ▪  (RefLink[NearestNeighbors, paclet:ref/method/NearestNeighbors])  ▪  (RefLink[DecisionTree, paclet:ref/method/DecisionTree])}})}})

({{(Spacer1), ({{(Spacer1)Related Links}, {(WebLink[DocumentationFooterIconsRelatedLinks-EIWL.png(Spacer1)An Elementary Introduction to the Wolfram Language, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])(WebLink[: Machine Learning, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])}})}})