({{, (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer])[f] 
represents a net layer that takes two input arrays and applies a function f to corresponding array elements.}})

OpenersNotesSection(Spacer1)Details and Options

(RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) is typically used inside (RefLink[NetGraph, paclet:ref/NetGraph]).

(RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) has an arbitrary number of input ports named 1, 2, etc.

Within a (RefLink[NetGraph, paclet:ref/NetGraph]), a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) can be connected using a single edge of the form {(Subscript[src, 1]),(Subscript[src, 2]),…}->threadlayer, where threadlayer is the name or index of the (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]), or as multiple separate edges given in the corresponding order, as (Subscript[src, 1])->threadlayer,(Subscript[src, 2])->threadlayer,…,(Subscript[src, n])->threadlayer.

In (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer])[f], the function f can be any one of the following: (RefLink[Plus, paclet:ref/Plus]), (RefLink[Subtract, paclet:ref/Subtract]), (RefLink[Times, paclet:ref/Times]), (RefLink[Divide, paclet:ref/Divide]), (RefLink[Power, paclet:ref/Power]), (RefLink[Min, paclet:ref/Min]), (RefLink[Max, paclet:ref/Max]), (RefLink[Clip, paclet:ref/Clip]).

In general, f can be any object that when applied to two arguments gives any combination of (RefLink[Plus, paclet:ref/Plus]), (RefLink[Subtract, paclet:ref/Subtract]), etc. together with (RefLink[Ramp, paclet:ref/Ramp]), (RefLink[LogisticSigmoid, paclet:ref/LogisticSigmoid]), (RefLink[Tanh, paclet:ref/Tanh]), (RefLink[ArcTan, paclet:ref/ArcTan]), (RefLink[ArcTanh, paclet:ref/ArcTanh]), (RefLink[Sin, paclet:ref/Sin]), (RefLink[Sinh, paclet:ref/Sinh]), (RefLink[ArcSin, paclet:ref/ArcSin]), (RefLink[ArcSinh, paclet:ref/ArcSinh]), (RefLink[Cos, paclet:ref/Cos]), (RefLink[Cosh, paclet:ref/Cosh]), (RefLink[ArcCos, paclet:ref/ArcCos]), (RefLink[ArcCosh, paclet:ref/ArcCosh]), (RefLink[Log, paclet:ref/Log]), (RefLink[Exp, paclet:ref/Exp]), (RefLink[Sqrt, paclet:ref/Sqrt]), (RefLink[Abs, paclet:ref/Abs]), (RefLink[Gamma, paclet:ref/Gamma]), (RefLink[LogGamma, paclet:ref/LogGamma]) and numbers.

The function f can also contain expressions of the form "name"[input], where name is one of the named functions ("ELU", "SELU", "SoftSign", etc.) accepted by (RefLink[ElementwiseLayer, paclet:ref/ElementwiseLayer]).

(RefLink[ThreadingLayer, paclet:ref/ThreadingLayer])[…][{(Subscript[in, 1]),(Subscript[in, 2]),…}] explicitly computes the output from applying the layer, which is effectively given by f[(Subscript[in, 1]),(Subscript[in, 2]),…].

When given a (RefLink[NumericArray, paclet:ref/NumericArray]) as input, the output will be a (RefLink[NumericArray, paclet:ref/NumericArray]).

(RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) exposes the following ports for use in (RefLink[NetGraph, paclet:ref/NetGraph]) etc.:

The option "Inputs"->n can be used to fix the number of inputs to (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]).

When it cannot be inferred from other layers in a larger net, the option "Output"->{(Subscript[d, 1]),(Subscript[d, 2]),…} can be used to fix the dimensions of the output of the (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]). All inputs will also have their dimensions fixed to the same value.

OpenersPrimaryExamplesSection(Spacer1)Examples  (10)

OpenersExampleSection0(Spacer1)Basic Examples  (3)

Create a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) using (RefLink[Times, paclet:ref/Times]) as the function to be threaded:

ThreadingLayer[Times]

Create a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) and apply it to a pair of inputs:

max=ThreadingLayer[Times]

Apply the layer to a pair of inputs:

max[{{1,2},{3,0}}]

Create a (RefLink[NetGraph, paclet:ref/NetGraph]) that contains a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]):

net=NetGraph[<|"thread"->ThreadingLayer[Times]|>,{{NetPort["x"],NetPort["y"]}->"thread"}]

Apply the layer to two vectors:

net[<|"x"->{1,2,3},"y"->{4,5,6}|>]

OpenersExampleSection1(Spacer1)Scope  (3)

Create a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) that takes a specific number of inputs:

ThreadingLayer[Max,"Inputs"->3]

Create a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) that takes two matrices as input, and apply it to data:

plus=ThreadingLayer[Plus,"Inputs"->2,"Output"->{2,3}]

mat1={{1,2,3},{4,5,6}};
mat2={{0.1,3,4},{1,2,3}};
plus[{mat1,mat2}]

Create a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) that uses a custom transformation to compute a spherical Gaussian:

gauss=ThreadingLayer[Exp[-(#1^2+#2^2)]&,"Output"->"Real"]

Evaluate the layer on two input vectors to get a vector of outputs:

gauss[{{0,1,2},{2,0,1}}]

Plot the output of the layer:

Plot3D[gauss[{x,y}],{x,-2,2},{y,-2,2},NormalsFunction->None]

OpenersExampleSection3(Spacer1)Applications  (1)

Define a hinge loss using a (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]):

HingeLossLayer[margin_]:=NetGraph[{ThreadingLayer[Ramp[Abs[#2-#1]-margin]&]},{{NetPort["Input"],NetPort["Target"]}->1},"Input"->"Real"]

Create a net that computes the hinge loss with a margin of 2:

hinge=HingeLossLayer[2]

When the target is within distance 2 of the input, the loss is zero:

hinge[<|"Input"->3,"Target"->2|>]

The loss increases linearly beyond a distance of 2:

hinge[<|"Input"->5,"Target"->2|>]

Plot the loss as a function of input for a fixed target of 2:

Plot[hinge[<|"Input"->x,"Target"->2|>],{x,-4,8}]

Perform linear regression using the hinge loss:

data={1->1,2->2,3->3,4->9,5->5};
trained=NetTrain[LinearLayer[],data,LossFunction->hinge,MaxTrainingRounds->1000]

Plot the solution:

dataPlot=ListPlot[List@@@data,PlotStyle->{Red,PointSize[Medium]}];
Show[dataPlot,Plot[trained[x],{x,0,5}]]

Compare the solutions obtained using the hinge loss, mean absolute and mean squared error:

{net1,net2,net3}=NetTrain[LinearLayer[],data,LossFunction->#1,MaxTrainingRounds->1000]&/@{MeanSquaredLossLayer[],MeanAbsoluteLossLayer[],HingeLossLayer[2]};

Show[
Plot[{Legended[net1[x],"squared"],Legended[net2[x],"absolute"],Legended[net3[x],"hinge"]},{x,0,5},PlotRange->All],
dataPlot]

OpenersExampleSection4(Spacer1)Possible Issues  (3)

(RefLink[ThreadingLayer, paclet:ref/ThreadingLayer]) cannot accept symbolic inputs:

ThreadingLayer[Times][{x,y}]

Certain choices of f can produce failures for inputs outside their domain:

ThreadingLayer[Divide][{{1,2},{3,0}}]

Some custom transformations are not supported:

ThreadingLayer[Mod[#1,#2]&]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[ElementwiseLayer, paclet:ref/ElementwiseLayer])  ▪  (RefLink[ThreadingLayer, paclet:ref/ThreadingLayer])  ▪  (RefLink[NetChain, paclet:ref/NetChain])  ▪  (RefLink[NetGraph, paclet:ref/NetGraph])  ▪  (RefLink[NetTrain, paclet:ref/NetTrain])  ▪  (RefLink[SoftmaxLayer, paclet:ref/SoftmaxLayer])  ▪  (RefLink[DropoutLayer, paclet:ref/DropoutLayer])}})}})

({{DocumentationFooterIconsRelatedTutorial.png, ({{Tutorials}, {▪(RefLinkPlain[Neural Networks in the Wolfram Language, paclet:tutorial/NeuralNetworksOverview])}})}})

({{DocumentationFooterIconsRelatedGuide.png, ({{Related Guides}, {▪(RefLinkPlain[Neural Networks, paclet:guide/NeuralNetworks])}})}})