({{, (RefLink[UtilityFunction, paclet:ref/UtilityFunction]) 
is an option for (RefLink[Predict, paclet:ref/Predict]), (RefLink[Classify, paclet:ref/Classify]), and related functions that specifies the utility value to assign to each possible pairing of actual and predicted values.}})

OpenersNotesSection(Spacer1)Details

With UtilityFunction->f, the utility value assigned to actual value (Subscript[v, a]) and predicted value (Subscript[v, p]) is taken to be f[(Subscript[v, a]),(Subscript[v, p])].

In the case of (RefLink[Classify, paclet:ref/Classify]), f is typically an association of associations, of the form <|(Subscript[class, 1])-><|(Subscript[class, 1])->(Subscript[u, 11]),(Subscript[class, 2])->(Subscript[u, 12])…|>,…|>. Here (Subscript[u, ij]) is the utility matrix, with i being the actual class and j the predicted class.

OpenersPrimaryExamplesSection(Spacer1)Examples  (3)

OpenersExampleSection0(Spacer1)Basic Examples  (2)

Train a classifier:

trainingset = {0->"healthy", 1->"healthy",2->"diseased",3->"healthy",4->"diseased",5->"diseased"};

c=Classify[trainingset]

For a new example, the most probable class is predicted:

c[1.5, "Probabilities"]

c[1.5]

Set the decision utility to penalize misclassification of class "diseased":

c[1.5,UtilityFunction-> <|
"healthy"-> <|"healthy"-> 1, "diseased"-> 0|>,"diseased"-> <|"healthy"-> -2,"diseased"->1|>
|>]

The decision utility can be specified in the classifier:

c=Classify[trainingset,UtilityFunction-> <|
"healthy"-> <|"healthy"-> 1, "diseased"-> 0|>,"diseased"-> <|"healthy"-> -2,"diseased"->1|>
|>]

c[1.5]

Use a utility function that includes an (RefLink[Indeterminate, paclet:ref/Indeterminate]) decision possibility (independently of the option (RefLink[IndeterminateThreshold, paclet:ref/IndeterminateThreshold])):

c[1.5,UtilityFunction-><|
"healthy"-> <|"healthy"-> 1, "diseased"-> 0, Indeterminate-> 0.7|>,"diseased"-> <|"healthy"-> 0,"diseased"->1, Indeterminate-> 0.9|>
|>]

Train a predictor:

trainingset = {1->2.2,2->4.4,3->5.5,4->7.7,5->9.4};

p=Predict[trainingset]

Visualize the probability density for a given example:

example=4;
pdf = PDF[p[example,"Distribution"]]

Plot[pdf[x],{x,5,10},PlotRange->All]

By default, the value with the highest probability density is predicted:

p[example]

Define a utility function that penalizes the predicted value being smaller than the actual value:

utility[a_,p_]:=-Piecewise[{{Exp[p-a],a<p},{Exp[3*(a-p)],a≥p}}]

Use this utility to predict the example:

p[example,UtilityFunction->utility]

The default utility function is the Dirac delta:

p[example,UtilityFunction->(DiracDelta[#2-#1]&)]

OpenersExampleSection2(Spacer1)Applications  (1)

Load the training set and test set of the "Mushroom" dataset:

trainingset = ExampleData[{"MachineLearning","Mushroom"},"TrainingData"];

testset = ExampleData[{"MachineLearning","Mushroom"},"TestData"];

Train a classifier on a part of the training set:

c=Classify[RandomSample[trainingset,100]]

Visualize the confusion matrix of the classifier on the test set:

ClassifierMeasurements[c,testset,"ConfusionMatrixPlot"]

Visualize the confusion matrix obtained when the classifier has a different value of (RefLink[UtilityFunction, paclet:ref/UtilityFunction]):

utility = <|
"edible"-><|"edible"-> 1,"poisonous"-> 0,Indeterminate-> 0.5|>,
"poisonous"-><|"edible"-> -10,"poisonous"->1, Indeterminate-> 0.8|>
|>;

ClassifierMeasurements[c,testset,"ConfusionMatrixPlot",UtilityFunction->utility]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[Classify, paclet:ref/Classify])  ▪  (RefLink[Predict, paclet:ref/Predict])  ▪  (RefLink[ClassifierFunction, paclet:ref/ClassifierFunction])  ▪  (RefLink[PredictorFunction, paclet:ref/PredictorFunction])  ▪  (RefLink[ClassPriors, paclet:ref/ClassPriors])  ▪  (RefLink[IndeterminateThreshold, paclet:ref/IndeterminateThreshold])}})}})