({{, (RefLink[LossFunction, paclet:ref/LossFunction]) 
is an option for (RefLink[NetTrain, paclet:ref/NetTrain]) that specifies how to compare actual and requested outputs from a neural net.}})

OpenersNotesSection(Spacer1)Details

In (RefLink[NetTrain, paclet:ref/NetTrain])[net,data,…], possible settings for (RefLink[LossFunction, paclet:ref/LossFunction]) include:

Layers that can be used as preconfigured loss layers include:

When specifying a single preconfigured loss layer with (RefLink[LossFunction, paclet:ref/LossFunction])->losslayer, the net being trained must have exactly one output port.

When specifying a loss for a specific port with (RefLink[LossFunction, paclet:ref/LossFunction])->"port"->lspec, the loss specification lspec can be a preconfigured loss layer or (RefLink[Automatic, paclet:ref/Automatic]).

When a loss layer is chosen automatically for a port, the loss layer to use is based on the layer within the net whose output is connected to the port, as follows:

If the attached loss layer has one input port ("Input"), it will be attached to the output of the net, and the keys of the training data should supply ports to feed the input of the net only (see figure below, left side).

If the attached loss layer has two input ports ("Input" and "Target"), the input will be attached to the output of the net and the target will be fed from the training data, using the name of the output port of the net (see figure below, center). Typically, this name is "Output", and so the usual case is training data of the form <|"Input"->{(Subscript[in, 1]),(Subscript[in, 2]),…},"Output"->{(Subscript[out, 1]),(Subscript[out, 2]),…}|>, which is often written as {(Subscript[in, 1]),(Subscript[in, 2]),…}->{(Subscript[out, 1]),(Subscript[out, 2]),…} or {(Subscript[in, 1])->(Subscript[out, 1]),(Subscript[in, 2])->(Subscript[out, 2]),…}.

If multiple such layers are connected, there should be one port in the training data to feed the target of each layer (see figure below, right side).

OpenersPrimaryExamplesSection(Spacer1)Examples  (4)

OpenersExampleSection0(Spacer1)Basic Examples  (4)

Train a simple net using (RefLink[MeanSquaredLossLayer, paclet:ref/MeanSquaredLossLayer]), the default loss applied to an output when it is not produced by a (RefLink[SoftmaxLayer, paclet:ref/SoftmaxLayer]):

net=LinearLayer[];
data={1->1.9,2->4.1,3->6.0,4->8.1};
trained=NetTrain[net,data]

Evaluate the trained net on a set of inputs:

trained[Range[4]]

Specify a different loss layer to attach to the output of the network. First create a loss layer:

loss=MeanAbsoluteLossLayer[]

The loss layer takes an input and a target and produces a loss:

loss[<|"Input"->5.0,"Target"->3.0|>]

Use this loss layer during training:

trained=NetTrain[net,data,LossFunction->loss]

Evaluate the trained net on a set of inputs:

trained[Range[4]]

Create a net that takes a vector of length 2 and produces one of the class labels (RefLink[Less, paclet:ref/Less]) or (RefLink[Greater, paclet:ref/Greater]):

net=NetChain[{LinearLayer[2],SoftmaxLayer[]},"Output"->NetDecoder[{"Class",{Less,Greater}}]]

(RefLink[NetTrain, paclet:ref/NetTrain]) will automatically use a (RefLink[CrossEntropyLossLayer, paclet:ref/CrossEntropyLossLayer]) object with the correct class encoder:

trained=NetTrain[net,{{1,2}->Less,{2,3}->Less,{4,2}->Greater,{3,1}->Greater}]

Evaluate the trained net on a set of inputs:

trained[{{3,5},{5,3}}]

Create an explicit loss layer that expects the targets to be in the form of probabilities for each class:

loss=CrossEntropyLossLayer["Probabilities"]

The training data should now consist of vectors of probabilities instead of symbolic classes:

trained=NetTrain[net,{{1,2}->{1.,0.},{2,3}->{1.,0.},{4,2}->{0.,1.},{3,1}->{0.,1.},{2,2}->{0.5,0.5},{3,3}->{0.5,0.5}},LossFunction->loss]

Evaluate the trained net on a set of inputs:

trained[{{3,5},{5,3}}]

Start with an "evaluation net" to be trained:

net=NetChain[{LinearLayer[8],Ramp,LinearLayer[4],Tanh,LinearLayer[{}]},"Input"->2]

Create a "loss net" that explicitly computes the loss of the evaluation net (here, the custom loss is equivalent to (RefLink[MeanSquaredLossLayer, paclet:ref/MeanSquaredLossLayer])):

lossNet=NetGraph[<|"net"->net,"loss"->ThreadingLayer[(#1-#2)^2&]|>,{{"net",NetPort["Target"]}->"loss"->NetPort["Loss"]}]

Train this net on some synthetic data, specifying that the output port named "Loss" should be interpreted as a loss:

data=Flatten@Table[{x,y}->Exp[-(x^2+y^2)],{x,-2,2,.01},{y,-2,2,.01}];
trainedLossNet=NetTrain[lossNet,data, LossFunction->"Loss"]

Obtain the trained "evaluation" network using (RefLink[NetExtract, paclet:ref/NetExtract]):

trainedNet=NetExtract[trainedLossNet,"net"]

(RefLink[Part, paclet:ref/Part]) syntax can also be used:

trainedNet=trainedLossNet[["net"]]

Plot the net's output on the plane:

Plot3D[trainedNet[{x,y}],{x,-2,2},{y,-2,2},NormalsFunction->None]

Create a training net that computes both an output and multiple explicit losses:

tnet=NetGraph[{3,Tanh,{},MeanAbsoluteLossLayer[],MeanSquaredLossLayer[]},{1->2->3->4,3->5,3->NetPort["Output"]},"Input"->"Real"]

This network requires an input and a target in order to produce the output and losses:

NetInitialize[tnet][<|"Input"->3,"Target"->9|>]

Train using a specific output as loss, ignoring all other outputs:

trained=NetTrain[tnet,{1->1,2->0,3->-1,4->0,5->1},LossFunction->"Loss2"];

Measure the loss on a single pair after training:

trained[<|"Input"->3,"Target"->-1|>]

Specify multiple losses to be trained jointly:

trained=NetTrain[tnet,{1->1,2->0,3->-1,4->0,5->1},LossFunction->{"Loss1","Loss2"}];

trained[<|"Input"->3,"Target"->-1|>]

Use (RefLink[NetTake, paclet:ref/NetTake]) to remove all but the desired output and input:

trained=NetTake[trained,{NetPort["Input"],NetPort["Output"]}]

Evaluate this network on a set of inputs:

trained[Range[5]]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[NetTrain, paclet:ref/NetTrain])  ▪  (RefLink[MeanAbsoluteLossLayer, paclet:ref/MeanAbsoluteLossLayer])  ▪  (RefLink[MeanSquaredLossLayer, paclet:ref/MeanSquaredLossLayer])  ▪  (RefLink[CrossEntropyLossLayer, paclet:ref/CrossEntropyLossLayer])  ▪  (RefLink[CTCLossLayer, paclet:ref/CTCLossLayer])  ▪  (RefLink[ContrastiveLossLayer, paclet:ref/ContrastiveLossLayer])  ▪  (RefLink[NMinimize, paclet:ref/NMinimize])}})}})

({{DocumentationFooterIconsRelatedTutorial.png, ({{Tutorials}, {▪(RefLinkPlain[Neural Networks in the Wolfram Language, paclet:tutorial/NeuralNetworksOverview])}})}})

({{DocumentationFooterIconsRelatedGuide.png, ({{Related Guides}, {▪(RefLinkPlain[Neural Networks, paclet:guide/NeuralNetworks])}})}})