Method for (RefLink[Predict, paclet:ref/Predict]), (RefLink[Classify, paclet:ref/Classify]) and (RefLink[LearnDistribution, paclet:ref/LearnDistribution]).

Use a decision tree to model class probabilities, value predictions or probability densities.

A decision tree is a flow chart–like structure in which each internal node represents a "test" on a feature, each branch represents the outcome of the test, and each leaf represents a class distribution, value distribution or probability density.

For (RefLink[Classify, paclet:ref/Classify]) and (RefLink[Predict, paclet:ref/Predict]), the tree is constructed using the CART algorithm.

For (RefLink[LearnDistribution, paclet:ref/LearnDistribution]), the splits are determined using an information criterion trading off the likelihood and the complexity of the model.

The following options can be given:

OpenersPrimaryExamplesSection(Spacer1)Examples  (7)

OpenersExampleSection0(Spacer1)Basic Examples  (3)

Train a predictor function on labeled examples:

p=Predict[{1,2,3,4}->{.3,.4,.6,9},Method->"DecisionTree"]

Look at the information about the predictor:

Information[p]

Extract option information that can be used for retraining:

Information[p,"MethodOption"]

Predict a new example:

p[1.3]

Generate some data and visualize it:

data=Table[x->Sin[x]+RandomVariate[NormalDistribution[0, .2]], {x, RandomReal[{-10, 10}, 400]}];
ListPlot[List@@@data]

Train a predictor function on it:

p=Predict[data,Method->"DecisionTree"]

Compare the data with the predicted values and look at the standard deviation:

Show[Plot[
{p[x],
p[x]+StandardDeviation[p[x,"Distribution"]],p[x]-StandardDeviation[p[x,"Distribution"]]},
{x,-2,6},
PlotStyle->{Blue,Gray,Gray},
Filling->{2->{3}},
Exclusions->False,
PerformanceGoal->"Speed",PlotLegends->{"Prediction","Confidence Interval"}],ListPlot[List@@@data,PlotStyle->Red,PlotLegends->{"Data"}]]

Learn a distribution using the method (StringTypeLink["DecisionTree", paclet:ref/method/DecisionTree]):

data=RandomVariate[NormalDistribution[],1000];

ld=LearnDistribution[data,Method->"DecisionTree"]

Visualize the PDF obtained:

Plot[PDF[ld,x],{x,-5,5},Filling->Bottom]

Obtain information about the distribution:

Information[ld]

OpenersExampleSection1(Spacer1)Options  (4)

OpenersExampleSubsection0(Spacer1)"DistributionSmoothing"  (2)

Use the "DistributionSmoothing" option to train a classifier:

c=Classify[{1,2,3,4,5, 6}->{1,1,3,3,1,3}, Method->{"DecisionTree","DistributionSmoothing"->.3}]

Use the mushrooms training set to train a classifier with the default value of "DistributionSmoothing":

data=ExampleData[{"MachineLearning","Mushroom"},"TrainingData"];

classifier=Classify[data,Method->"DecisionTree"];

Train a second classifier using a large "DistributionSmoothing":

smoothed=Classify[data,Method->{"DecisionTree","DistributionSmoothing"->6}]

Compare the probabilities for examples from a test set:

testdata=ExampleData[{"MachineLearning","Mushroom"},"TestData"];

sample=RandomSample[testdata,4];
Dataset@<|"AutomaticClassifier"->
classifier[sample[[All, 1]],"Probabilities"], 
"SmoothedClassifier"->smoothed[sample[[All, 1]],"Probabilities"]|>

Using a large value of "DistributionSmoothing" typically reduces the size of the tree. This can be observed by counting the number of leaves in each tree:

leaves1=Normal[classifier[[1,"Model","Tree","LeafValues"]]];
leaves2=Normal[smoothed[[1,"Model","Tree","LeafValues"]]];
Length/@{leaves1, leaves2}

OpenersExampleSubsection1(Spacer1)"FeatureFraction"  (2)

Use the "FeatureFraction" option to train a classifier:

c= Classify[{{1, 2.3, 4, 5.3},{2, 2.3, 2.4, 5},{2, 2.3, 2.4, 5},{1, 3, 4, -5.2},{2, -5, -3.2, 5}, {2, 1.3, -8.1, 3.3}}->{1,1,3,3,1,3}, Method->{"DecisionTree","FeatureFraction"->.5}]

Use the mushrooms training set to train two classifiers with different values of "FeatureFraction":

data=ExampleData[{"MachineLearning","Mushroom"},"TrainingData"];

c1=Classify[data, Method->{"DecisionTree","FeatureFraction"->1}]

c2=Classify[data,Method->{"DecisionTree","FeatureFraction"->.1}]

Look at the accuracy of these classifiers on a test set:

testdata=ExampleData[{"MachineLearning","Mushroom"},"TestData"];

ClassifierMeasurements[c1,testdata,"Accuracy"]

ClassifierMeasurements[c2,testdata,"Accuracy"]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[Classify, paclet:ref/Classify])  ▪  (RefLink[Predict, paclet:ref/Predict])  ▪  (RefLink[LearnDistribution, paclet:ref/LearnDistribution])  ▪  (RefLink[ClassifierFunction, paclet:ref/ClassifierFunction])  ▪  (RefLink[PredictorFunction, paclet:ref/PredictorFunction])  ▪  (RefLink[LearnedDistribution, paclet:ref/LearnedDistribution])  ▪  (RefLink[Information, paclet:ref/Information])}, {Related Machine Learning Methods:  (RefLink[RandomForest, paclet:ref/method/RandomForest])  ▪  (RefLink[GradientBoostedTrees, paclet:ref/method/GradientBoostedTrees])  ▪  (RefLink[NearestNeighbors, paclet:ref/method/NearestNeighbors])  ▪  (RefLink[KernelDensityEstimation, paclet:ref/method/KernelDensityEstimation])  ▪  (RefLink[SupportVectorMachine, paclet:ref/method/SupportVectorMachine])  ▪  (RefLink[GaussianProcess, paclet:ref/method/GaussianProcess])  ▪  (RefLink[NaiveBayes, paclet:ref/method/NaiveBayes])}})}})

({{(Spacer1), ({{(Spacer1)Related Links}, {(WebLink[DocumentationFooterIconsRelatedLinks-EIWL.png(Spacer1)An Elementary Introduction to the Wolfram Language, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])(WebLink[: Machine Learning, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])}})}})