Check validity of custom layer
checkLayer(
checks the validity of a custom layer using generated data of the sizes in
layer,validInputSize)validInputSize. For layers with a single input, set
validInputSize to a typical size of input data to the
layer. For layers with multiple inputs, set validInputSize to a
cell array of typical sizes, where each element corresponds to a layer input.
checkLayer(
specifies additional options using one or more name-value pairs.layer,validInputSize,Name,Value)
Check the validity of the example custom layer preluLayer.
Define a custom PReLU layer. To create this layer, save the file preluLayer.m in the current folder.
Create an instance of the layer and check that it is valid using checkLayer. Set the valid input size to the typical size of a single observation input to the layer. For a single input, the layer expects observations of size h-by-w-by-c, where h, w, and c are the height, width, and number of channels of the previous layer output, respectively.
Specify validInputSize as the typical size of an input array.
layer = preluLayer(20,'prelu');
validInputSize = [5 5 20];
checkLayer(layer,validInputSize)Skipping multi-observation tests. To enable tests with multiple observations, specify the 'ObservationDimension' option. For 2-D image data, set 'ObservationDimension' to 4. For 3-D image data, set 'ObservationDimension' to 5. For sequence data, set 'ObservationDimension' to 2. Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward ......... Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 9 Passed, 0 Failed, 0 Incomplete, 12 Skipped. Time elapsed: 0.11901 seconds.
The results show the number of passed, failed, and skipped tests. If you do not specify the 'ObservationsDimension' option, or do not have a GPU, then the function skips the corresponding tests.
Check Multiple Observations
For multi-observation input, the layer expects an array of observations of size h-by-w-by-c-by-N, where h, w, and c are the height, width, and number of channels, respectively, and N is the number of observations.
To check the layer validity for multiple observations, specify the typical size of an observation and set 'ObservationDimension' to 4.
layer = preluLayer(20,'prelu'); validInputSize = [5 5 20]; checkLayer(layer,validInputSize,'ObservationDimension',4)
Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward .......... ... Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 13 Passed, 0 Failed, 0 Incomplete, 8 Skipped. Time elapsed: 0.069311 seconds.
In this case, the function does not detect any issues with the layer.
Check code generation compatibility of the custom layer codegenPreluLayer.
Define a custom PReLU layer with code generation support. To create this layer, save the file codegenPreluLayer.m in the current folder.
Create an instance of the layer and check its validity using checkLayer. Specify the valid input size to be the size of a single observation of typical input to the layer. The layer expects 4-D array inputs, where the first three dimensions correspond to the height, width, and number of channels of the previous layer output, and the fourth dimension corresponds to the observations.
Specify the typical size of the input of an observation and set the 'ObservationDimension' option to 4. To check for code generation compatibility set the 'CheckCodegenCompatibility' option to true.
layer = codegenPreluLayer(20,'prelu'); validInputSize = [24 24 20]; checkLayer(layer,validInputSize,'ObservationDimension',4,'CheckCodegenCompatibility',true)
Skipping GPU tests. No compatible GPU device found. Running nnet.checklayer.TestLayerWithoutBackward .......... ....... Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 17 Passed, 0 Failed, 0 Incomplete, 4 Skipped. Time elapsed: 0.56219 seconds.
Here, the function does not detect any issues with the layer.
layer — Custom layernnet.layer.Layer object | nnet.layer.ClassificationLayer object | nnet.layer.RegressionLayer objectCustom layer, specified as an nnet.layer.Layer object,
nnet.layer.ClassificationLayer object, or
nnet.layer.RegressionLayer object. For an example
showing how to define your own custom layer, see Define Custom Deep Learning Layer with Learnable Parameters.
validInputSize — Valid input sizesValid input sizes of the layer, specified as a vector of positive integers or cell array of vectors of positive integers.
For layers with a single input, specify
validInputSize as a vector of integers corresponding to the dimensions of
the input data. For example, [5 5 10]
corresponds to valid input data of size 5-by-5-by-10.
For layers with multiple inputs, specify
validInputSize as a cell array of vectors, where each vector corresponds
to a layer input and the elements of the vectors correspond to the dimensions of the
corresponding input data. For example, {[24 24 20],[24 24
10]} corresponds to the valid input sizes of two
inputs, where 24-by-24-by-20 is a valid input size for the first
input and 24-by-24-by-10 is a valid input size for the second
input.
For more information, see Layer Input Sizes.
For large input sizes, the gradient checks take longer to run. To speed up the tests, specify a smaller valid input size.
Example: [5 5 10]
Example: {[24 24 20],[24 24 10]}
Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | cell
Specify optional
comma-separated pairs of Name,Value arguments. Name is
the argument name and Value is the corresponding value.
Name must appear inside quotes. You can specify several name and value
pair arguments in any order as
Name1,Value1,...,NameN,ValueN.
'ObservationDimension',4 sets the observation dimension
to 4'ObservationDimension' — Observation dimensionObservation dimension, specified as the comma-separated pair
consisting of 'ObservationDimension' and a positive
integer.
The observation dimension specifies which dimension of the layer input data corresponds to observations. For example, if the layer expects input data is of size h-by-w-by-c-by-N, where h, w, and c correspond to the height, width, and number of channels of the input data, respectively, and N corresponds to the number of observations, then the observation dimension is 4. For more information, see Layer Input Sizes.
If you specify the observation dimension, then the
checkLayer function checks that the layer
functions are valid using generated data with mini-batches of size 1 and
2. If you do not specify the observation dimension, then the function
skips the corresponding tests.
Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64
'CheckCodegenCompatibility' — Flag to enable code generation testsfalse (default) | trueFlag to enable code generation tests, specified as the comma-separated
pair consisting of 'CheckCodegenCompatibility' and
true or false.
If 'CheckCodegenCompatibility' is
true, then you must set the 'ObservationDimension' option.
Data Types: logical
For each layer, the valid input size and the observation dimension depend on the output of the previous layer.
For intermediate layers (layers of type nnet.layer.Layer),
the valid input size and the observation dimension depend on the type of data
input to the layer. For layers with a single input, specify
validInputSize as a vector of integers corresponding to the dimensions of
the input data.For layers with multiple inputs, specify
validInputSize as a cell array of vectors, where each vector corresponds
to a layer input and the elements of the vectors correspond to the dimensions of the
corresponding input data.
For large input sizes, the gradient checks take longer to run. To speed up the tests, specify a smaller valid input size.
| Layer Input | Input Size | Observation Dimension |
|---|---|---|
| 2-D images | h-by-w-by-c-by-N, where h, w, and c correspond to the height, width, and number of channels of the images respectively, and N is the number of observations. | 4 |
| 3-D images | h-by-w-by-d-by-c-by-N, where h, w, d, and c correspond to the height, width, depth, and number of channels of the 3-D images respectively, and N is the number of observations. | 5 |
| Vector sequences | c-by-N-by-S, where c is the number of features of the sequences, N is the number of observations, and S is the sequence length. | 2 |
| 2-D image sequences | h-by-w-by-c-by-N-by-S, where h, w, and c correspond to the height, width, and number of channels of the images respectively, N is the number of observations, and S is the sequence length. | 4 |
| 3-D image sequences | h-by-w-by-d-by-c-by-N-by-S, where h, w, d, and c correspond to the height, width, depth, and number of channels of the 3-D images respectively, N is the number of observations, and S is the sequence length. | 5 |
For example, for 2-D image classification problems, set
validInputSize to [h w c], where
h, w, and c
correspond to the height, width, and number of channels of the images,
respectively, and 'ObservationDimension' to
4.
Code generation supports intermediate layers with 2-D image input only.
For output layers (layers of type
nnet.layer.ClassificationLayer or
nnet.layer.RegressionLayer), set
validInputSize to the typical size of a single input
observation Y to the layer.
For classification problems, the valid input size and the observation
dimension of Y depend on the type of problem:
| Classification Task | Input Size | Observation Dimension |
|---|---|---|
| 2-D image classification | 1-by-1-by-K-by-N, where K is the number of classes and N is the number of observations. | 4 |
| 3-D image classification | 1-by-1-by-1-by-K-by-N, where K is the number of classes and N is the number of observations. | 5 |
| Sequence-to-label classification | K-by-N, where K is the number of classes and N is the number of observations. | 2 |
| Sequence-to-sequence classification | K-by-N-by-S, where K is the number of classes, N is the number of observations, and S is the sequence length. | 2 |
For example, for 2-D image classification problems, set
validInputSize to [1 1 K], where
K is the number of classes, and
'ObservationDimension' to 4.
For regression problems, the dimensions of Y also depend on
the type of problem. The following table describes the dimensions of
Y.
| Regression Task | Input Size | Observation Dimension |
|---|---|---|
| 2-D image regression | 1-by-1-by-R-by-N, where R is the number of responses and N is the number of observations. | 4 |
| 2-D Image-to-image regression | h-by-w-by-c-by-N,
where h, w, and
c are the height, width, and number of channels
of the output respectively, and N is the number of
observations. | 4 |
| 3-D image regression | 1-by-1-by-1-by-R-by-N, where R is the number of responses and N is the number of observations. | 5 |
| 3-D Image-to-image regression | h-by-w-by-d-by-c-by-N,
where h, w, d,
and c are the height, width, depth, and number of
channels of the output respectively, and N is the
number of observations. | 5 |
| Sequence-to-one regression | R-by-N, where R is the number of responses and N is the number of observations. | 2 |
| Sequence-to-sequence regression | R-by-N-by-S, where R is the number of responses, N is the number of observations, and S is the sequence length. | 2 |
For example, for 2-D image regression problems, set
validInputSize to [1 1 R], where
R is the number of responses, and
'ObservationDimension' to 4.
The checkLayer function checks the validity of a custom layer
by performing a series of tests, described in these tables. For more information on
the tests used by checkLayer, see Check Custom Layer Validity.
The checkLayer function uses these tests to check the validity of custom
intermediate layers (layers of type nnet.layer.Layer).
| Test | Description |
|---|---|
functionSyntaxesAreCorrect | The syntaxes of the layer functions are correctly defined. |
predictDoesNotError | predict does not error. |
forwardDoesNotError | When specified, |
forwardPredictAreConsistentInSize | When |
backwardDoesNotError | When specified, backward does not error. |
backwardIsConsistentInSize | When
|
predictIsConsistentInType | The outputs of |
forwardIsConsistentInType | When |
backwardIsConsistentInType | When |
gradientsAreNumericallyCorrect | When backward is specified, the gradients computed
in backward are consistent with the numerical
gradients. |
backwardPropagationDoesNotError | When backward is not specified, the derivatives
can be computed using automatic differentiation. |
codegenPragmaDefinedInClassDef | The pragma "%#codegen" for code generation is
specified in class file. |
checkForSupportedLayerPropertiesForCodegen | The layer properties support code generation. |
predictIsValidForCodeGeneration | predict is valid for code generation. |
The tests predictIsConsistentInType, forwardIsConsistentInType, and backwardIsConsistentInType also check for GPU compatibility. To execute the layer functions on a GPU, the functions must support inputs and outputs of type gpuArray with the underlying data type single.
The checkLayer function uses these tests to check the
validity of custom output layers (layers of type
nnet.layer.ClassificationLayer or
nnet.layer.RegressionLayer).
| Test | Description |
|---|---|
forwardLossDoesNotError | forwardLoss does not error. |
backwardLossDoesNotError | backwardLoss does not error. |
forwardLossIsScalar | The output of forwardLoss is scalar. |
backwardLossIsConsistentInSize | When backwardLoss is specified, the output of
backwardLoss is consistent in size:
dLdY is the same size as the predictions
Y. |
forwardLossIsConsistentInType | The output of |
backwardLossIsConsistentInType | When |
gradientsAreNumericallyCorrect | When backwardLoss is specified, the gradients computed
in backwardLoss are numerically correct. |
backwardPropagationDoesNotError | When backwardLoss is not specified, the derivatives
can be computed using automatic differentiation. |
The forwardLossIsConsistentInType and
backwardLossIsConsistentInType tests also check for GPU compatibility. To
execute the layer functions on a GPU, the functions must support inputs and outputs of type
gpuArray with the underlying data type single.
analyzeNetwork | trainingOptions | trainNetwork
You have a modified version of this example. Do you want to open this example with your edits?