sequenceFoldingLayer
Sequence folding layer
Description
A sequence folding layer converts a batch of image sequences to a batch of images. Use a sequence folding layer to perform convolution operations on time steps of image sequences independently.
To use a sequence folding layer, you must connect the miniBatchSize
output to the miniBatchSize input of the corresponding sequence unfolding
layer. For an example, see Create Network for Video Classification.
Creation
Description
layer = sequenceFoldingLayer
Properties
Examples
Create Sequence Folding Layer
Create a sequence folding layer with name the 'fold1'.
layer = sequenceFoldingLayer('Name','fold1')
layer =
SequenceFoldingLayer with properties:
Name: 'fold1'
NumOutputs: 2
OutputNames: {'out' 'miniBatchSize'}
Create Network for Video Classification
Create a deep learning network for data containing sequences of images, such as video and medical image data.
To input sequences of images into a network, use a sequence input layer.
To apply convolutional operations independently to each time step, first convert the sequences of images to an array of images using a sequence folding layer.
To restore the sequence structure after performing these operations, convert this array of images back to image sequences using a sequence unfolding layer.
To convert images to feature vectors, use a flatten layer.
You can then input vector sequences into LSTM and BiLSTM layers.
Define Network Architecture
Create a classification LSTM network that classifies sequences of 28-by-28 grayscale images into 10 classes.
Define the following network architecture:
A sequence input layer with an input size of
[28 28 1].A convolution, batch normalization, and ReLU layer block with 20 5-by-5 filters.
An LSTM layer with 200 hidden units that outputs the last time step only.
A fully connected layer of size 10 (the number of classes) followed by a softmax layer and a classification layer.
To perform the convolutional operations on each time step independently, include a sequence folding layer before the convolutional layers. LSTM layers expect vector sequence input. To restore the sequence structure and reshape the output of the convolutional layers to sequences of feature vectors, insert a sequence unfolding layer and a flatten layer between the convolutional layers and the LSTM layer.
inputSize = [28 28 1]; filterSize = 5; numFilters = 20; numHiddenUnits = 200; numClasses = 10; layers = [ ... sequenceInputLayer(inputSize,'Name','input') sequenceFoldingLayer('Name','fold') convolution2dLayer(filterSize,numFilters,'Name','conv') batchNormalizationLayer('Name','bn') reluLayer('Name','relu') sequenceUnfoldingLayer('Name','unfold') flattenLayer('Name','flatten') lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm') fullyConnectedLayer(numClasses, 'Name','fc') softmaxLayer('Name','softmax') classificationLayer('Name','classification')];
Convert the layers to a layer graph and connect the miniBatchSize output of the sequence folding layer to the corresponding input of the sequence unfolding layer.
lgraph = layerGraph(layers); lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
View the final network architecture using the plot function.
figure plot(lgraph)
Extended Capabilities
See Also
bilstmLayer | classifyAndUpdateState | flattenLayer | gruLayer | lstmLayer | predictAndUpdateState | resetState | sequenceInputLayer | sequenceUnfoldingLayer
Topics
- Classify Videos Using Deep Learning
- Sequence Classification Using Deep Learning
- Time Series Forecasting Using Deep Learning
- Sequence-to-Sequence Classification Using Deep Learning
- Visualize Activations of LSTM Network
- Long Short-Term Memory Networks
- Specify Layers of Convolutional Neural Network
- Set Up Parameters and Train Convolutional Neural Network
- Deep Learning in MATLAB
- List of Deep Learning Layers