({{, (RefLink[NetStateObject, paclet:ref/NetStateObject])[net] 
creates an object derived from net that represents a neural net with additional stored state information that is updated when the net is applied to data.}, {, (RefLink[NetStateObject, paclet:ref/NetStateObject])[net,seed]
creates an object in which additional stored state information is initialized using seed.}})

OpenersNotesSection(Spacer1)Details

(RefLink[NetStateObject, paclet:ref/NetStateObject])[…][data] updates stored state information in the (RefLink[NetStateObject, paclet:ref/NetStateObject]).

State information is associated with the state ports of recurrent net layers such as (RefLink[LongShortTermMemoryLayer, paclet:ref/LongShortTermMemoryLayer]).

(RefLink[NetStateObject, paclet:ref/NetStateObject]) will not store the state of layers whose state ports are initialized from other layers in a (RefLink[NetGraph, paclet:ref/NetGraph]).

When a seed is not provided, initial values for recurrent states will consist of arrays of zeros.

The current value of the stored states is given by (RefLink[NetExtract, paclet:ref/NetExtract])[NetStateObject[…],"States"].

OpenersPrimaryExamplesSection(Spacer1)Examples  (7)

OpenersExampleSection0(Spacer1)Basic Examples  (3)

Create a recurrent net:

net=NetInitialize@BasicRecurrentLayer[1,"Input"->{"Varying",1}]

Create a state object from the recurrent net:

sobj=NetStateObject@net

Evaluate the state object on some data:

sobj[{{1},{3},{2},{2}}]

Due to the presence of the stored states, the behavior of the state object can change between evaluations, even on the same input:

sobj[{{1}}]

sobj[{{1}}]

Create a state object with a specified initial state:

net=NetInitialize@BasicRecurrentLayer[3,"Input"->{"Varying",2}];
sobj=NetStateObject[net,<|{"State"}:>{1,-1,0}|>]

Create a classifier that predicts the next element of a sequence:

chain=NetChain[{EmbeddingLayer[4],BasicRecurrentLayer[3],SequenceLastLayer[],SoftmaxLayer[]},"Output"->NetDecoder[{"Class", {1,2,3}}]]

Train the classifier on a set of input sequences:

trained=NetTrain[chain,{{1,2}->3,{2,3}->1,{3,1}->2}]

Create a state object and use it to efficiently generate a maximum-likelihood sequence, starting from a single 1:

sobj=NetStateObject[trained]

NestList[sobj[{#}]&,1,8]

OpenersExampleSection2(Spacer1)Applications  (1)

Training an English character-level language model. First, create 300,000 training examples of 25 characters each from two novels:

corpus=StringJoin[ResourceData["The American"]," " , ResourceData["Don Quixote - English"]];

sampleCorpus[corpus_,len_,num_]:=Module[
{positions=RandomInteger[{len+1,StringLength[corpus]-1},num]},
Thread[StringTake[corpus,{#-len,#}&/@positions]->StringPart[corpus,positions+1]]];
trainingData=sampleCorpus[corpus,25,300000];

The data is of the form of a classification problem: given a sequence of characters, predict the next one. A sample of the data:

RandomSample[trainingData,5]//Map[InputForm]//TableForm

Obtain the list of all characters in the text:

characters=Union[Characters[corpus]]

Define a net that takes in a string of characters and returns a prediction for the next character:

net=NetChain[{
UnitVectorLayer[],
GatedRecurrentLayer[128],
GatedRecurrentLayer[128],
SequenceLastLayer[],
LinearLayer[Length[characters]],
SoftmaxLayer[]},
"Input"->NetEncoder[{"Characters",characters}],
"Output"->NetDecoder[{"Class",characters}]
];

Train the net. This can take up to an hour on a CPU; it is recommended to specify a GPU if possible:

trainedNet=NetTrain[net,trainingData,ValidationSet->Scaled[0.1],BatchSize->64,TargetDevice->"GPU"]

Export["trainedLanguageModel.wlnet",trainedNet];

Predict the next character, given a sequence of characters:

trainedNet["Why are w"]

Generate 100 characters of text, given starting text:

generate[net_,start_,len_]:=
StringJoin@NestList[NetStateObject[net],start,len]

generate[trainedNet,"Why are w",100]

One can get more interesting text by sampling from the probability distribution of predictions:

generateSample[net_,start_,len_]:=Block[{sobj=NetStateObject[net]},
StringJoin@NestList[sobj[#,"RandomSample"]&,start,len]]

generateSample[trainedNet,"Why are w",100]

OpenersExampleSection3(Spacer1)Properties & Relations  (3)

If the initial value of a recurrent layer's state is provided by a connection in a (RefLink[NetGraph, paclet:ref/NetGraph]), that state will not be stored by (RefLink[NetStateObject, paclet:ref/NetStateObject]).

Create a graph that uses a connection to provide the initial value of the state of a (RefLink[BasicRecurrentLayer, paclet:ref/BasicRecurrentLayer]):

graph=NetInitialize@NetGraph[
{BasicRecurrentLayer[2,"Input"->{"Varying",2}]},
{NetPort["State"]->NetPort[1,"State"]}]

This graph cannot be used inside a (RefLink[NetStateObject, paclet:ref/NetStateObject]), as there are no states left to store:

NetStateObject[graph]

The current value of the stored states can be obtained using (RefLink[NetExtract, paclet:ref/NetExtract]).

First create a (RefLink[NetStateObject, paclet:ref/NetStateObject]):

net=NetInitialize@BasicRecurrentLayer[2,"Input"->{"Varying",2}];
sobj=NetStateObject[net]

Apply the object to some data:

sobj[{{1,2},{2,0},{0,1}}]

Extract the current value of the states:

NetExtract[sobj,"States"]

For recurrent nets, using a (RefLink[NetStateObject, paclet:ref/NetStateObject]) is equivalent to manually keeping track of the recurrent states via (RefLink[NetPort, paclet:ref/NetPort])[(RefLink[All, paclet:ref/All]),"States"].

To see this, create a classifier that predicts the next element of a sequence:

chain=NetChain[{EmbeddingLayer[3],BasicRecurrentLayer[3],SequenceLastLayer[],SoftmaxLayer[]},"Output"->NetDecoder["Class"]]

Train the classifier on a set of input sequences:

trained=NetTrain[chain,{{1,2}->3,{2,3}->1,{3,1}->2}]

Create a state object and use it to efficiently generate a maximum-likelihood sequence, starting from a single 1:

sobj=NetStateObject[trained]

NestList[sobj[{#}]&,1,8]

Generate from the trained net using (RefLink[NetPort, paclet:ref/NetPort])[(RefLink[All, paclet:ref/All]),"States"] to set and get the recurrent states, which yields the same result:

state=None;
NestList[Function[
{output,state}=Values@trained[<|"Input"->{#},NetPort[All,"States"]->state|>,{"Output",NetPort[All,"States"]}];
output],1,8]

({{DocumentationFooterIconsRelatedFunction.png, ({{See Also}, {(RefLink[NetChain, paclet:ref/NetChain])  ▪  (RefLink[NetGraph, paclet:ref/NetGraph])  ▪  (RefLink[BasicRecurrentLayer, paclet:ref/BasicRecurrentLayer])  ▪  (RefLink[GatedRecurrentLayer, paclet:ref/GatedRecurrentLayer])  ▪  (RefLink[LongShortTermMemoryLayer, paclet:ref/LongShortTermMemoryLayer])  ▪  (RefLink[NetFoldOperator, paclet:ref/NetFoldOperator])}})}})

({{DocumentationFooterIconsRelatedTutorial.png, ({{Tutorials}, {▪(RefLinkPlain[Neural Networks in the Wolfram Language, paclet:tutorial/NeuralNetworksOverview])}})}})

({{DocumentationFooterIconsRelatedGuide.png, ({{Related Guides}, {▪(RefLinkPlain[Neural Networks, paclet:guide/NeuralNetworks])}})}})