Import an ONNX network as a function. The network contains ONNX operators that are not supported by Deep Learning Toolbox layers. You can use the imported model function for deep learning tasks, such as prediction and transfer learning.

Download and install the Deep Learning Toolbox Converter for ONNX Model Format support package. You can enter importONNXFunction at the command line to check if the support package is installed. If it is not installed, then the function provides a link to the required support package in the Add-On Explorer. To install the support package, click the link, and then click Install.

Specify the file to import as shufflenet with operator set 9 from the ONNX Model Zoo. shufflenet is a convolutional neural network that is trained on images from the ImageNet database.

modelfile = 'shufflenet-9.onnx';

A recommended practice is to try to import the network by using importONNXNetwork. If importONNXNetwork is unable to import the network because some of the network layers are not supported, you can import the network as layers by using importONNXLayers, or as a function by using importONNXFunction.

Import the shufflenet network as layers. The software generates placeholder layers in place of the unsupported layers.

lgraph = importONNXLayers(modelfile,'OutputLayerType','classification');

Warning: Unable to import some ONNX operators, because they are not supported. They have been replaced by placeholder layers. To find these layers, call the function findPlaceholderLayers on the returned object.

4 operators(s)	:	Average pooling layer in ONNX file does not include padding in the average. This may cause small numeric differences between the ONNX and MATLAB network outputs.
32 operators(s)	:	The Reshape operator is supported only when it performs a flattening operation.
16 operators(s)	:	The operator 'Transpose' is not supported.

To import the ONNX network as a function, which can support most ONNX operators, call importONNXFunction.

Find the placeholder layers and display the number of placeholder layers.

indPlaceholderLayers = findPlaceholderLayers(lgraph);
numel(indPlaceholderLayers)

ans = 48

You must replace the 48 placeholder layers to use lgraph for deep learning tasks, such as prediction.

Instead, import the network as a function to generate a model function that you can readily use for deep learning tasks.

params = importONNXFunction(modelfile,'shufflenetFcn')

OpsetVersion = 9
A function 'shufflenetFcn' containing the imported ONNX network has been saved to the current directory.
To learn how to use this function, type: help shufflenetFcn

params = 
  ONNXParameters with properties:

             Learnables: [1×1 struct]
          Nonlearnables: [1×1 struct]
                  State: [1×1 struct]
          NumDimensions: [1×1 struct]
    NetworkFunctionName: 'shufflenetFcn'

importONNXFunction returns the ONNXParameters object params, which contains the network parameters, and the model function shufflnetFcn, which contains the network architecture. importONNXFunction saves shufflenetFcn in the current folder. You can open the model function to view or edit the network architecture by using open shufflenetFcn.

Import an ONNX network as a function, and use the pretrained network to predict the class label of an input image.

Specify the file to import as shufflenet with operator set 9 from the ONNX Model Zoo. shufflenet is a convolutional neural network that is trained on more than a million images from the ImageNet database. As a result, the network has learned rich feature representations for a wide range of images. The network can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many animals.

modelfile = 'shufflenet-9.onnx';

Import the pretrained ONNX network as a function by using importONNXFunction, which returns an ONNXParamaters object that contains the network parameters. The function also creates a new model function in the current folder that contains the network architecture. Specify the name of the model function as shufflenetFcn.

params = importONNXFunction(modelfile,'shufflenetFcn');

OpsetVersion = 9
A function 'shufflenetFcn' containing the imported ONNX network has been saved to the current directory.
To learn how to use this function, type: help shufflenetFcn

Read the image you want to classify and display the size of the image. The image is 792-by-1056 pixels and has three color channels (RGB).

I = imread('peacock.jpg');
size(I)

ans = 1×3

         792        1056           3

Resize the image to the input size of the network. Show the image.

I = imresize(I,[224 224]);
imshow(I)

The inputs to shufflenet require further preprocessing (for more details, see ShuffleNet in ONNX Model Zoo). Rescale the image. Normalize the image by subtracting the training images mean and dividing by the training images standard deviation.

I = rescale(I,0,1);

meanIm = [0.485 0.456 0.406];
stdIm = [0.229 0.224 0.225];
I = (I - reshape(meanIm,[1 1 3]))./reshape(stdIm,[1 1 3]);

imshow(I)

Import the class names from squeezenet, which is also trained with images from the ImageNet database.

net = squeezenet;
ClassNames = net.Layers(end).ClassNames;

Calculate the class probabilities by specifying the image to classify I and the ONNXParameters object params as input arguments to the model function shufflenetFcn.

scores = shufflenetFcn(I,params);

Find the class index with the highest probability. Display the predicted class for the input image and the corresponding classification score.

indMax = find(scores==max(scores));
ClassNames(indMax)

ans = 1×1 cell array
    {'peacock'}

scoreMax = scores(indMax)

scoreMax = 0.7517

Import the alexnet convolution neural network as a function and fine-tune the pretrained network with transfer learning to perform classification on a new collection of images.

This example uses several helper functions. To view the code for these functions, see Helper Functions.

Unzip and load the new images as an image datastore. imageDatastore automatically labels the images based on folder names and stores the data as an ImageDatastore object. An image datastore enables you to store large image data, including data that does not fit in memory, and efficiently read batches of images during training of a convolutional neural network. Specify the mini-batch size.

unzip('MerchData.zip');
miniBatchSize = 8;
imds = imageDatastore('MerchData', ...
    'IncludeSubfolders',true, ...
    'LabelSource','foldernames',...
    'ReadSize', miniBatchSize);

This data set is small, containing 75 training images. Display some sample images.

numImages = numel(imds.Labels);
idx = randperm(numImages,16);
figure
for i = 1:16
    subplot(4,4,i)
    I = readimage(imds,idx(i));
    imshow(I)
end

Extract the training set and one-hot encode the categorical classification labels.

XTrain = readall(imds);
XTrain = single(cat(4,XTrain{:}));
YTrain_categ = categorical(imds.Labels);
YTrain = onehotencode(YTrain_categ,2)';

Determine the number of classes in the data.

classes = categories(YTrain_categ);
numClasses = numel(classes)

numClasses = 5

AlexNet is a convolutional neural network that is trained on more than a million images from the ImageNet database. As a result, the network has learned rich feature representations for a wide range of images. The network can classify images into 1000 object categories, such as keyboard, mouse, pencil, and many animals.

Import the pretrained alexnet network as a function.

alexnetONNX()
params = importONNXFunction('alexnet.onnx','alexnetFcn')

A function containing the imported ONNX network has been saved to the file alexnetFcn.m.
To learn how to use this function, type: help alexnetFcn.

params = 
  ONNXParameters with properties:

             Learnables: [1×1 struct]
          Nonlearnables: [1×1 struct]
                  State: [1×1 struct]
          NumDimensions: [1×1 struct]
    NetworkFunctionName: 'alexnetFcn'

params is an ONNXParameters object that contains the network parameters. alexnetFcn is a model function that contains the network architecture. importONNXFunction saves alexnetFcn in the current folder.

Calculate the classification accuracy of the pretrained network on the new training set.

accuracyBeforeTraining = getNetworkAccuracy(XTrain,YTrain,params);
fprintf('%.2f accuracy before transfer learning\n',accuracyBeforeTraining);

0.01 accuracy before transfer learning

The accuracy is very low.

Display the learnable parameters of the network. These parameters, for example the weights (W) and bias (B) of convolution and fully connected layers, are updated by the network during training. Nonlearnable parameters remain constant during training.

params.Learnables

ans = struct with fields:
    data_Mean: [227×227×3 dlarray]
      conv1_W: [11×11×3×96 dlarray]
      conv1_B: [96×1 dlarray]
      conv2_W: [5×5×48×256 dlarray]
      conv2_B: [256×1 dlarray]
      conv3_W: [3×3×256×384 dlarray]
      conv3_B: [384×1 dlarray]
      conv4_W: [3×3×192×384 dlarray]
      conv4_B: [384×1 dlarray]
      conv5_W: [3×3×192×256 dlarray]
      conv5_B: [256×1 dlarray]
        fc6_W: [6×6×256×4096 dlarray]
        fc6_B: [4096×1 dlarray]
        fc7_W: [1×1×4096×4096 dlarray]
        fc7_B: [4096×1 dlarray]
        fc8_W: [1×1×4096×1000 dlarray]
        fc8_B: [1000×1 dlarray]

The last two learnable parameters of the pretrained network are configured for 1000 classes. The parameters fc8_W and fc8_B must be fine-tuned for the new classification problem. Transfer the parameters to classify 5 classes by initializing them.

params.Learnables.fc8_B = rand(5,1);
params.Learnables.fc8_W = rand(1,1,4096,5);

Freeze all the parameters of the network to convert them to nonlearnable parameters. Because you do not need to compute the gradients of the frozen layers, freezing the weights of many initial layers can significantly speed up network training.

params = freezeParameters(params,'all');

Unfreeze the last two parameters of the network to convert them to learnable parameters.

params = unfreezeParameters(params,'fc8_W');
params = unfreezeParameters(params,'fc8_B');

Now the network is ready for training. Initialize the training progress plot.

plots = "training-progress";
if plots == "training-progress"
    figure
    lineLossTrain = animatedline;
    xlabel("Iteration")
    ylabel("Loss")
end

Specify the training options.

velocity = [];
numEpochs = 5;
miniBatchSize = 16;
numObservations = size(YTrain,2);
numIterationsPerEpoch = floor(numObservations./miniBatchSize);
initialLearnRate = 0.01;
momentum = 0.9;
decay = 0.01;

Train the network.

iteration = 0;
start = tic;
executionEnvironment = "cpu"; % Change to "gpu" to train on a GPU.

% Loop over epochs.
for epoch = 1:numEpochs
    
    % Shuffle data.
    idx = randperm(numObservations);
    XTrain = XTrain(:,:,:,idx);
    YTrain = YTrain(:,idx);
    
    % Loop over mini-batches.
    for i = 1:numIterationsPerEpoch
        iteration = iteration + 1;
        
        % Read mini-batch of data.
        idx = (i-1)*miniBatchSize+1:i*miniBatchSize;
        X = XTrain(:,:,:,idx);        
        Y = YTrain(:,idx);
        
        % If training on a GPU, then convert data to gpuArray.
        if (executionEnvironment == "auto" && canUseGPU) || executionEnvironment == "gpu"
            X = gpuArray(X);         
        end
        
        % Evaluate the model gradients and loss using dlfeval and the
        % modelGradients function.
        [gradients,loss,state] = dlfeval(@modelGradients,X,Y,params);
        params.State = state;
        
        % Determine learning rate for time-based decay learning rate schedule.
        learnRate = initialLearnRate/(1 + decay*iteration);
        
        % Update the network parameters using the SGDM optimizer.
        [params.Learnables,velocity] = sgdmupdate(params.Learnables,gradients,velocity);
        
        % Display the training progress.
        if plots == "training-progress"
            D = duration(0,0,toc(start),'Format','hh:mm:ss');
            addpoints(lineLossTrain,iteration,double(gather(extractdata(loss))))
            title("Epoch: " + epoch + ", Elapsed: " + string(D))
            drawnow
        end
    end
end

Calculate the classification accuracy of the network after fine-tuning.

accuracyAfterTraining = getNetworkAccuracy(XTrain,YTrain,params);
fprintf('%.2f accuracy after transfer learning\n',accuracyAfterTraining);

0.99 accuracy after transfer learning

function accuracy = getNetworkAccuracy(X,Y,onnxParams)

N = size(X,4);
Ypred = alexnetFcn(X,onnxParams,'Training',false);

[~,YIdx] = max(Y,[],1);
[~,YpredIdx] = max(Ypred,[],1);
numIncorrect = sum(abs(YIdx-YpredIdx) > 0);
accuracy = 1 - numIncorrect/N;

end

function [grad, loss, state] = modelGradients(X,Y,onnxParams)

[y,state] = alexnetFcn(X,onnxParams,'Training',true);
loss = crossentropy(y,Y,'DataFormat','CB');
grad = dlgradient(loss,onnxParams.Learnables);

end

function alexnetONNX()
    
exportONNXNetwork(alexnet,'alexnet.onnx');

end