Method for (RefLink[Classify, paclet:ref/Classify]).

Models class probabilities by finding a hyperplane that separates the training data into two classes using a maximum-margin hyperplane.

Support vector machines are binary classifiers. A kernel function is used to extract features from the examples. At training time, the method finds the maximum-margin hyperplane that separates classes. The multiclass classification problem is reduced to a set of binary classification problems (using a one-vs.-one or a one-vs.-all strategy). The current implementation uses the LibSVM framework in the back end.

The option "KernelType" allows you to choose the type of kernel to use. Possible settings for "KernelType" include:

The kernel "RadialBasisFunction" takes the form:

K((Subscript[x, 1]),(Subscript[x, 2]))=exp(-γ*((Abs[(Subscript[x, 1])-(Subscript[x, 2])])^2))

The kernel "Polynomial" takes the form:

K((Subscript[x, 1]),(Subscript[x, 2]))=γ*(((c+(Subscript[x, 1]).(Subscript[x, 2])))^d)

The kernel "Sigmoid" takes the form:

K((Subscript[x, 1]),(Subscript[x, 2]))=tanh(c+γ* (Subscript[x, 1]).(Subscript[x, 2]))

The kernel "Linear" takes the form:

(Subscript[x, 1]).(Subscript[x, 2])

The following options can be given:

Possible settings for "MulticlassStrategy" include:

The "GammaScalingParameter" controls the influence of the support vectors. Large values of gamma mean small radius of influence.

The "PolynomialDegree" option is specific to the polynomial kernel type.

The "MulticlassStrategy" option is used to generalize binary classifiers to a multiclass ones. The "OneVersusOne" strategy tests each class again each other, while the "OneVersusAll" strategy only test each class against the rest of the training set.

OpenersPrimaryExamplesSection(Spacer1)Examples  (7)

OpenersExampleSection0(Spacer1)Basic Examples  (2)

Train a classifier function on labeled examples:

c=Classify[{1,2,3,4}->{1,1,2,2},Method->"SupportVectorMachine"]

Obtain information about the classifier:

Information[c]

Classify a new example:

c[1.3]

Generate some data that is not linearly separable:

cluster1= RandomVariate[MultinormalDistribution[{0,0},IdentityMatrix[2]],200];
cluster2=MapThread[{#1 Cos[#2],#1 Sin[#2]}&, { RandomReal[{4,5},200],RandomReal[{0,2 Pi},200]}];

clusters={cluster1,cluster2};

Visualize it:

ListPlot[clusters,PlotStyle->Darker@{Yellow,Blue,Green}]

Train a classifier on this dataset:

c=Classify[<|Yellow->clusters[[1]],Blue->clusters[[2]]|>, Method->"SupportVectorMachine"]

Plot the training set and the probability distribution of each class as a function of the features:

Show[
Plot3D[{
c[{x,y},"Probability"->Yellow],
c[{x,y},"Probability"->Blue]
},
{x,-4,4},{y,-5,4},
 Exclusions->None],
ListPointPlot3D[Map[Append[#,1]&,clusters,{2}], PlotStyle->{Yellow,Blue}]]

OpenersExampleSection1(Spacer1)Options  (5)

OpenersExampleSubsection0(Spacer1)"KernelType"  (2)

Train a classifier using a specific "KernelType":

Classify[<|1->{1,23,1.3, 4}, 2->{-4, -3.2, -4, -5}|>, Method->{"SupportVectorMachine", "KernelType"->"Linear"}]

Train two classifiers using different instances of "KernelType":

data={->2,->5,->8,->0,->2,->7,->5,->1,->3,->0,->3,->9,->6,->2,->8,->2,->0,->6,->6,->1,->1,->7,->8,->5,->0,->4,->7,->6,->0,->2,->5,->3,->1,->5,->6,->7,->5,->4,->1,->9,->3,->6,->8,->0,->9,->3,->0,->3,->7,->4,->4,->3,->8,->0,->4,->1,->3,->7,->6,->4,->7,->2,->7,->2,->5,->2,->0,->9,->8,->9,->8,->1,->6,->4,->8,->5,->8,->0,->6,->7,->4,->5,->8,->4,->3,->1,->5,->1,->9,->9,->9,->2,->4,->7,->3,->1,->9,->2,->9,->6};

clinear=Classify[data, Method->{"SupportVectorMachine", "KernelType"->"Linear"}];

cpolynomial=Classify[data, Method->{"SupportVectorMachine", "KernelType"->"Polynomial"}];

Compare their performance:

clinear[{,,,,,,,,,}]
cpolynomial[{,,,,,,,,,}]

OpenersExampleSubsection1(Spacer1)"MulticlassStrategy"  (1)

Use the "FisherIris" dataset to train two classifiers with different "MulticlassStrategy" options:

trainingset=ExampleData[{"MachineLearning","FisherIris"}, "TrainingData"];

{c1,c2}=Classify[trainingset, Method->{"SupportVectorMachine","MulticlassStrategy"->#}]&/@{"OneVersusOne","OneVersusAll"};

Look at their accuracy on a test set. The "OneVersusOne" option typically performs better.

testset=ExampleData[{"MachineLearning","FisherIris"},"TestData"];

ClassifierMeasurements[#,testset,"Accuracy"]&/@{c1,c2}

OpenersExampleSubsection2(Spacer1)"GammaScalingParameter"  (1)

Train a classifier with a specific value for the "GammaScalingParameter" suboption:

c=Classify[<|1->{1,23,1.3, 4}, 2->{-4, -3.2, -4, -5}|>, Method->{"SupportVectorMachine", "GammaScalingParameter" -> 3}]

The "GammaScalingParameter" controls the influence of the support vectors.

Generate some data and visualize it:

data1=RandomVariate[MultinormalDistribution[{0,0}, IdentityMatrix[2]], 200];
data2 =Transpose[{ #radius *Cos[#angle],#radius* Sin[#angle]}&@<|"radius"->RandomReal[{5, 8},200], "angle"->RandomReal[{0, 2 Pi},200]|>];
trainingset=<|4->data1,2->data2|>;

ListPlot[{data1,data2}]

Train two classifiers by changing the "GammaScalingParameter":

c2=Classify[trainingset, Method->{"SupportVectorMachine","GammaScalingParameter"->2}]

c20=Classify[trainingset, Method->{"SupportVectorMachine","GammaScalingParameter"->20}]

Look at how they perform on a test set to see how the radius of influence has changed:

test=RandomReal[{-8, 8}, {2000, 2}];

Grid[{{ListPlot[{Style@@@Transpose[{test,Hue/@( c2[test]/4)}], data1}], 
ListPlot[{Style@@@Transpose[{test,Hue/@( c20[test]/4)}], data1}]}}, Frame->All]

OpenersExampleSubsection3(Spacer1)"PolynomialDegree"  (1)

Use the "Mushroom" training set to train two classifiers using different degrees for the polynomial kernel type:

trainingset=ExampleData[{"MachineLearning","Mushroom"},"TrainingData"];

{c1,c2}=Classify[trainingset, Method->{"SupportVectorMachine","KernelType"->"Polynomial", "PolynomialDegree"->#}]&/@{3,7};

Compare the corresponding training times:

ClassifierInformation[#,"TrainingTime"]&/@{c1 ,c2}

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[Classify, paclet:ref/Classify])  ▪  (RefLink[ClassifierFunction, paclet:ref/ClassifierFunction])  ▪  (RefLink[ClassifierMeasurements, paclet:ref/ClassifierMeasurements])  ▪  (RefLink[Predict, paclet:ref/Predict])  ▪  (RefLink[SequencePredict, paclet:ref/SequencePredict])  ▪  (RefLink[ClusterClassify, paclet:ref/ClusterClassify])  ▪    ▪    ▪    ▪    ▪    ▪    ▪    ▪  }, {Related Machine Learning Methods:  (RefLink[DecisionTree, paclet:ref/method/DecisionTree])  ▪  (RefLink[LogisticRegression, paclet:ref/method/LogisticRegression])  ▪  (RefLink[GradientBoostedTrees, paclet:ref/method/GradientBoostedTrees])  ▪  (RefLink[Markov, paclet:ref/method/Markov])  ▪  (RefLink[NaiveBayes, paclet:ref/method/NaiveBayes])  ▪  (RefLink[NearestNeighbors, paclet:ref/method/NearestNeighbors])  ▪  (RefLink[NeuralNetwork, paclet:ref/method/NeuralNetwork])  ▪  (RefLink[RandomForest, paclet:ref/method/RandomForest])}})}})

({{(Spacer1), ({{(Spacer1)Related Links}, {(WebLink[DocumentationFooterIconsRelatedLinks-EIWL.png(Spacer1)An Elementary Introduction to the Wolfram Language, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])(WebLink[: Machine Learning, https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html])}})}})