Prerequisites

For supported versions of libraries and for information about setting up environment variables, see Prerequisites for Deep Learning with MATLAB Coder. This example is not supported for MATLAB online.

This example uses the DAG network MobileNet-v2 to perform image classification with the ARM® Compute Library. The pretrained MobileNet-v2 network for MATLAB is available in the Deep Learning Toolbox Model for MobileNet-v2 Network support package.

When you generate code that uses the ARM Compute Library and a hardware support package, codegen generates code on the host computer, copies the generated files to the target hardware, and builds the executable on the target hardware.

Configure Code Generation for the mobilenet_predict Function

The mobilenet_predict function calls the predict method of the MobileNet-v2 network object on an input image and returns the prediction score output. The function calls coder.updateBuildInfo to specify linking options for the generated makefile.

type mobilenet_predict

function out = mobilenet_predict(in)

persistent net;
opencv_linkflags = '`pkg-config --cflags --libs opencv`';
coder.updateBuildInfo('addLinkFlags',opencv_linkflags);
if isempty(net)
    net = coder.loadDeepLearningNetwork('mobilenetv2', 'mobilenet');
end

out = net.predict(in);

end

Create a C++ code generation configuration object.

cfg = coder.config('exe');
cfg.TargetLang = 'C++';

Specify Use of the ARM Compute Library. The ARM Compute Library provides optimized functionality for the Raspberry Pi hardware. To generate code that uses the ARM Compute Library, create a coder.ARMNEONConfig object. Specify the version of the ARM Compute Library installed on your Raspberry Pi and the architecture of the Raspberry Pi. Attach the deep learning configuration object to the code generation configuration object.

dlcfg = coder.DeepLearningConfig('arm-compute');
supportedVersions = dlcfg.getARMComputeSupportedVersions;
dlcfg.ArmArchitecture = 'armv7';
dlcfg.ArmComputeVersion = '19.05';
cfg.DeepLearningConfig = dlcfg;

Create a Connection to the Raspberry Pi

Use the MATLAB Support Package for Raspberry Pi Hardware function raspi to create a connection to the Raspberry Pi. In this code, replace:

r = raspi('raspiname','username','password');

Configure Code Generation Hardware Parameters for Raspberry Pi

Create a coder.Hardware object for the Raspberry Pi and attach it to the code generation configuration object.

hw = coder.hardware('Raspberry Pi');
cfg.Hardware = hw;

Specify a build folder on the Raspberry Pi:

buildDir = '~/remoteBuildDir';
cfg.Hardware.BuildDir = buildDir;

Provide a C++ Main File

Specify the main file main_mobilenet.cpp in the code generation configuration object. The file calls the generated C++ code for the mobilenet_predict function. The file reads the input image, passes the data to the generated function calls, retrieves the predictions on the image, and prints the prediction scores to a file.

cfg.CustomSource = 'main_mobilenet.cpp';

Generate the Executable Program on the Raspberry Pi

Generate C++ code. When you use codegen with the MATLAB Support Package for Raspberry PI Hardware, the executable is built on the Raspberry Pi.

For code generation, you must set the Environment Variables ARM_COMPUTELIB and LD_LIBRARY_PATH on the Raspberry Pi.

codegen -config cfg mobilenet_predict -args {ones(224, 224, 3,'single')} -report

Fetch the Generated Executable Folder

To test the generated code on the Raspberry Pi, copy the input image to the generated code folder. You can find this folder manually or by using the raspi.utils.getRemoteBuildDirectory API. This function lists the folders of the binary files that are generated by using codegen. Assuming that the binary is found in only one folder, enter:

applicationDirPaths = raspi.utils.getRemoteBuildDirectory('applicationName','mobilenet_predict');
targetDirPath = applicationDirPaths{1}.directory;

Copy Example Files to the Raspberry Pi

To copy files required to run the executable program, use putFile.

r.putFile('peppers_raspi_mobilenet.png',targetDirPath);

Run the Executable Program on the Raspberry Pi

Run the executable program on the Raspberry Pi from MATLAB and direct the output back to MATLAB.

exeName = 'mobilenet_predict.elf';
argsforexe = ' peppers_raspi_mobilenet.png '; % Provide the input image;
command = ['cd ' targetDirPath ';sudo ./' exeName argsforexe];
output = system(r,command);

Get the Prediction Scores for the 1000 Output Classes of the Network

outputfile = [targetDirPath, '/output.txt'];
r.getFile(outputfile);

Map the Prediction Scores to Labels and Display Output

Map the top five prediction scores to the corresponding labels in the trained network, and display the output.

type mapPredictedScores_mobilenet

%% Map the Prediction Scores to Labels and Display Output
net = mobilenetv2;
ClassNames = net.Layers(end).ClassNames;

%% Read the classification
fid = fopen('output.txt') ;
S = textscan(fid,'%s');
fclose(fid) ;
S = S{1} ;
predict_scores = cellfun(@(x)str2double(x), S);

%% Remove NaN values that were strings
predict_scores(isnan(predict_scores))=[];
[val,indx] = sort(predict_scores, 'descend');
scores = val(1:5)*100;
top5labels = ClassNames(indx(1:5));

%% Display classification labels on the image
im = imread('peppers_raspi_mobilenet.png');
im = imresize(im, [224 224]);
outputImage = zeros(224,400,3, 'uint8');
for k = 1:3
    outputImage(:,177:end,k) = im(:,:,k);
end
scol = 1;
srow = 1;
outputImage = insertText(outputImage, [scol, srow], 'Classification with MobileNetv2', 'TextColor', 'w','FontSize',20, 'BoxColor', 'black');
srow = srow + 30;
for k = 1:5
    outputImage = insertText(outputImage, [scol, srow], [top5labels{k},' ',num2str(scores(k), '%2.2f'),'%'], 'TextColor', 'w','FontSize',15, 'BoxColor', 'black');
    srow = srow + 25;
end
imshow(outputImage);