({{, (RefLink[ValidationSet, paclet:ref/ValidationSet]) 
is an option for (RefLink[Predict, paclet:ref/Predict]), (RefLink[Classify, paclet:ref/Classify]), (RefLink[NetTrain, paclet:ref/NetTrain]), and related functions that specifies the validation set to be used during the training phase.}})

OpenersNotesSection(Spacer1)Details

With ValidationSet->data, model and hyperparameter selections are done by testing performance on data. data can be given in any format allowed for the training set.

With ValidationSet->(RefLink[Automatic, paclet:ref/Automatic]), cross-validation methods on the original data supplied to (RefLink[Predict, paclet:ref/Predict]), (RefLink[Classify, paclet:ref/Classify]), etc. will be used instead.

ValidationSet->data is typically used when the data in the training set and the data that one wishes to predict or classify come from different sources.

OpenersPrimaryExamplesSection(Spacer1)Examples  (3)

OpenersExampleSection0(Spacer1)Basic Examples  (2)

Train a classifier:

trainingset = {1->"a",2->"a",3-> "b",4-> "a", 5-> "b", 6-> "b"};

c=Classify[trainingset,Method-> "NearestNeighbors"]

The automatically selected validation set yielded a 1-neighbor classifier:

ClassifierInformation[c, "NeighborsNumber"]

Specify the validation set explicitly:

c=Classify[trainingset,Method-> "NearestNeighbors",ValidationSet -> {1.6-> "a", 2.6-> "a",3.4-> "b", 5.1-> "b"}]

The explicit validation set yields a 2-neighbor classifier:

ClassifierInformation[c, "NeighborsNumber"]

Train a predictor with a specified validation set:

trainingset= {1->1,2->2,3-> 3,4-> 4, 5-> 5, 6-> 6};

p1=Predict[trainingset,ValidationSet->{1.5->1.6,2.5->2.4,3.5-> 3.7}]

Obtain the L2 regularization coefficient of the trained predictor:

PredictorInformation[p1,"L2Regularization"]

Train a predictor without a specified validation set:

p2=Predict[trainingset]

A different L2 regularization coefficient has been selected:

PredictorInformation[p2,"L2Regularization"]

OpenersExampleSection2(Spacer1)Applications  (1)

Load handwritten digits from the MNIST dataset:

MNISTdigits=ExampleData[{"MachineLearning","MNIST"},"TestData"];

Construct a set of digits from your own writing:

personaldigits={->6,->5,->8,->8,->6,->5,->0,->8,->4,->0,->9,->3,->1,->7,->6,->0,->1,->5,->7,->6,->3,->9,->2,->6,->8,->7,->0,->5,->7,->9,->4,->4,->9,->0,->5,->4,->4,->7,->3,->9,->0,->1,->4,->3,->1,->8,->4,->5,->1,->8,->7,->8,->9,->3,->5,->6,->2,->6,->2,->9,->4,->2,->8,->7,->1,->6,->2,->3,->2,->0,->2,->0,->2,->5,->3,->9,->7,->3};

Train a classifier that uses the MNIST digits for training and your own digits for validating models and hyperparameters:

c=Classify[MNISTdigits, ValidationSet->personaldigits]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[Classify, paclet:ref/Classify])  ▪  (RefLink[Predict, paclet:ref/Predict])  ▪  (RefLink[NetTrain, paclet:ref/NetTrain])}})}})

({{DocumentationFooterIconsRelatedGuide.png, ({{Related Guides}, {▪(RefLinkPlain[Neural Networks, paclet:guide/NeuralNetworks])}})}})