Deep Learning Prediction with Intel MKL-DNN

This example uses:

This example shows how to use codegen to generate code for an image classification application that uses deep learning on Intel® processors. The generated code takes advantage of the Intel Math Kernel Library for Deep Neural Networks (MKL-DNN). First, the example generates a MEX function that runs prediction by using the ResNet-50 image classification network. Then, the example builds a static library and compiles it with a main file that runs prediction using the ResNet-50 image classification network.

Prerequisites

Xeon processor with support for Intel Advanced Vector Extensions 2 (Intel AVX2) instructions
Intel Math Kernel Library for Deep Neural Networks (MKL-DNN)
Open Source Computer Vision Library (OpenCV) v3.1
Environment variables for Intel MKL-DNN and OpenCV
MATLAB® Coder™, for C++ code generation.
The support package MATLAB Coder Interface for Deep Learning.
Deep Learning Toolbox™, for using the DAGNetwork object
The Support package Deep Learning Toolbox Model for ResNet-50 Network support package, for using the pretrained ResNet network.

For more information, see Prerequisites for Deep Learning with MATLAB Coder (MATLAB Coder).

This example is supported on Linux® and Windows® platforms and not supported for MATLAB Online.

`resnet_predict` Function

This example uses the DAG network ResNet-50 to show image classification with MKL-DNN. A pretrained ResNet-50 model for MATLAB is available in the support package Deep Learning Toolbox Model for ResNet-50 Network. To download and install the support package, use the Add-On Explorer. See Get and Manage Add-Ons (MATLAB).

The resnet_predict function loads the ResNet-50 network into a persistent network object. On subsequent calls to the function, the persistent object is reused.

type resnet_predict

% Copyright 2018 The MathWorks, Inc.

function out = resnet_predict(in) 
%#codegen

% A persistent object mynet is used to load the series network object.
% At the first call to this function, the persistent object is constructed and
% setup. When the function is called subsequent times, the same object is reused 
% to call predict on inputs, avoiding reconstructing and reloading the
% network object.

persistent mynet;

if isempty(mynet)
    % Call the function resnet50 that returns a DAG network
    % for ResNet-50 model.
    mynet = coder.loadDeepLearningNetwork('resnet50','resnet');
end

% pass in input   
out = mynet.predict(in);

Generate MEX Code for the `resnet_predict` Function

To generate a MEX function from the resnet_predict.m function, use codegen with a deep learning configuration object created for the MKL-DNN library. Attach the deep learning configuration object to the MEX code generation configuration object that you pass to codegen.

 cfg = coder.config('mex');
 cfg.TargetLang = 'C++';
 cfg.DeepLearningConfig = coder.DeepLearningConfig('mkldnn');
 codegen -config cfg resnet_predict -args {ones(224,224,3,'single')} -report

Code generation successful: To view the report, open('codegen\mex\resnet_predict\html\report.mldatx').

Call `predict` on a Test Image

im = imread('peppers.png');
im = imresize(im, [224,224]);
imshow(im);
predict_scores = resnet_predict_mex(single(im));

Map the top five prediction scores to words in the synset dictionary.

fid = fopen('synsetWords.txt');
synsetOut = textscan(fid,'%s', 'delimiter', '\n');
synsetOut = synsetOut{1};
fclose(fid);
[val,indx] = sort(predict_scores, 'descend');
scores = val(1:5)*100;
top5labels = synsetOut(indx(1:5));

Display the top five classification labels on the image.

outputImage = zeros(224,400,3, 'uint8');
for k = 1:3
    outputImage(:,177:end,k) = im(:,:,k);
end

scol = 1;
srow = 1;
outputImage = insertText(outputImage, [scol, srow], 'Classification with ResNet-50', 'TextColor', 'w','FontSize',20, 'BoxColor', 'black');
srow = srow + 30;
for k = 1:5
    outputImage = insertText(outputImage, [scol, srow], [top5labels{k},' ',num2str(scores(k), '%2.2f'),'%'], 'TextColor', 'w','FontSize',15, 'BoxColor', 'black');
    srow = srow + 25;
end

imshow(outputImage);

Clear the static network object from memory.

clear mex;

Generate a Static Library for the `resnet_predict` Function

To generate a static library from the resnet_predict.m function, use codegen with a deep learning configuration object created for the MKL-DNN library. Attach the deep learning configuration object to the code generation configuration object that you pass to codegen.

cfg = coder.config('lib');
cfg.TargetLang = 'C++';
cfg.DeepLearningConfig = coder.DeepLearningConfig('mkldnn');
codegen -config cfg resnet_predict -args {ones(224,224,3,'single')} -report
%
codegendir = fullfile(pwd, 'codegen', 'lib', 'resnet_predict');

Code generation successful: To view the report, open('codegen\lib\resnet_predict\html\report.mldatx').

`main_resnet.cpp` File

The main file is used to generate an executable from the static library created by the codegen command. The main file reads the input image, runs prediction on the image, and displays the classification labels on the image.

type main_resnet.cpp

/* Copyright 2018 The MathWorks, Inc. */

#include "resnet_predict.h"

#include <stdio.h>
#include <string.h>
#include <math.h>
#include <iostream>
#include "opencv2/opencv.hpp"
using namespace cv;

int readData(void* inputBuffer, char* inputImage) {

    Mat inpImage, intermImage;
    inpImage = imread(inputImage, 1);
    Size size(224, 224);
    resize(inpImage, intermImage, size);
    if (!intermImage.data) {
        printf(" No image data \n ");
        exit(1);
    }
    float* input = (float*)inputBuffer;

    for (int j = 0; j < 224 * 224; j++) {
        // BGR to RGB
        input[2 * 224 * 224 + j] = (float)(intermImage.data[j * 3 + 0]);
        input[1 * 224 * 224 + j] = (float)(intermImage.data[j * 3 + 1]);
        input[0 * 224 * 224 + j] = (float)(intermImage.data[j * 3 + 2]);
    }
    return 1;
}

#if defined(WIN32) || defined(_WIN32) || defined(__WIN32) || defined(_WIN64)

int cmpfunc(void* r, const void* a, const void* b) {
    float x = ((float*)r)[*(int*)b] - ((float*)r)[*(int*)a];
    return (x > 0 ? ceil(x) : floor(x));
}
#else

int cmpfunc(const void* a, const void* b, void* r) {
    float x = ((float*)r)[*(int*)b] - ((float*)r)[*(int*)a];
    return (x > 0 ? ceil(x) : floor(x));
}

#endif

void top(float* r, int* top5) {
    int t[1000];
    for (int i = 0; i < 1000; i++) {
        t[i] = i;
    }
#if defined(WIN32) || defined(_WIN32) || defined(__WIN32) || defined(_WIN64)
    qsort_s(t, 1000, sizeof(int), cmpfunc, r);
#else
    qsort_r(t, 1000, sizeof(int), cmpfunc, r);
#endif
    top5[0] = t[0];
    top5[1] = t[1];
    top5[2] = t[2];
    top5[3] = t[3];
    top5[4] = t[4];
    return;
}


int prepareSynset(char synsets[1000][100]) {
    FILE* fp1 = fopen("synsetWords.txt", "r");
    if (fp1 == 0) {
        return -1;
    }

    for (int i = 0; i < 1000; i++) {
        if (fgets(synsets[i], 100, fp1) != NULL)
            ;
        strtok(synsets[i], "\n");
    }
    fclose(fp1);
    return 0;
}

void writeData(float* output, char synsetWords[1000][100], Mat &frame) {
    int top5[5], j;
    
    top(output, top5);
    
    copyMakeBorder(frame, frame, 0, 0, 400, 0, BORDER_CONSTANT, CV_RGB(0,0,0));
    char strbuf[50];
    sprintf(strbuf, "%4.1f%% %s", output[top5[0]]*100, synsetWords[top5[0]]);
    putText(frame, strbuf, cvPoint(30,80), CV_FONT_HERSHEY_DUPLEX, 1.0, CV_RGB(220,220,220), 1);
    sprintf(strbuf, "%4.1f%% %s", output[top5[1]]*100, synsetWords[top5[1]]);
    putText(frame, strbuf, cvPoint(30,130), CV_FONT_HERSHEY_DUPLEX, 1.0, CV_RGB(220,220,220), 1);
    sprintf(strbuf, "%4.1f%% %s", output[top5[2]]*100, synsetWords[top5[2]]);
    putText(frame, strbuf, cvPoint(30,180), CV_FONT_HERSHEY_DUPLEX, 1.0, CV_RGB(220,220,220), 1);
    sprintf(strbuf, "%4.1f%% %s", output[top5[3]]*100, synsetWords[top5[3]]);
    putText(frame, strbuf, cvPoint(30,230), CV_FONT_HERSHEY_DUPLEX, 1.0, CV_RGB(220,220,220), 1);
    sprintf(strbuf, "%4.1f%% %s", output[top5[4]]*100, synsetWords[top5[4]]);
    putText(frame, strbuf, cvPoint(30,280), CV_FONT_HERSHEY_DUPLEX, 1.0, CV_RGB(220,220,220), 1);

}

// Main function
int main(int argc, char* argv[]) {
    int n = 1;
    char synsetWords[1000][100];
    
    namedWindow("Classification with ResNet-50",CV_WINDOW_NORMAL);
    resizeWindow("Classification with ResNet-50",440,224);

    Mat im;
    im = imread(argv[1], 1);
    
    float* ipfBuffer = (float*)calloc(sizeof(float), 224*224*3);
    
    float* opBuffer = (float*)calloc(sizeof(float), 1000);
    if (argc != 2) {
        printf("Input image missing \nSample Usage-./resnet_exe image.png\n");
        exit(1);
    }
    if (prepareSynset(synsetWords) == -1) {
        printf("ERROR: Unable to find synsetWords.txt\n");
        return -1;
    }

    //read input imaget to the ipfBuffer
    readData(ipfBuffer, argv[1]);
    
    //run prediction on image stored in ipfBuffer
    resnet_predict(ipfBuffer, opBuffer);
    
    //write predictions on input image
    writeData(opBuffer, synsetWords, im);

    //show predictions on input image
    imshow("Classification with ResNet-50", im);
    waitKey(5000);
    destroyWindow("Classification with ResNet-50");
    return 0;
}

Build and Run the Executable

Build the executable based on the target platform. On a Windows platform, this example uses Microsoft® Visual Studio® 2017 for C++.

if ispc
    setenv('MATLAB_ROOT', matlabroot);
    system('make_mkldnn_win17.bat');
    system('resnet.exe peppers.png');
else
    setenv('MATLAB_ROOT', matlabroot);
    system('make -f Makefile_mkldnn_linux.mk');
    system('./resnet_exe peppers.png');
end

The results from the MEX function might not match the results from the generated static library function due to differences in the version of the library that is used to read the input image file. The image that is passed to the MEX function is read using the version that MATLAB ships. The image that is passed to the static library function is read using the version that OpenCV uses.

Documentation

Deep Learning Prediction with Intel MKL-DNN

Prerequisites

`resnet_predict` Function

Generate MEX Code for the `resnet_predict` Function

Call `predict` on a Test Image

Generate a Static Library for the `resnet_predict` Function

`main_resnet.cpp` File

Build and Run the Executable

Related Topics

Deep Learning Toolbox Documentation

Support

Documentation

Deep Learning Prediction with Intel MKL-DNN

Prerequisites

resnet_predict Function

Generate MEX Code for the resnet_predict Function

Call predict on a Test Image

Generate a Static Library for the resnet_predict Function

main_resnet.cpp File

Build and Run the Executable

Related Topics

Deep Learning Toolbox Documentation

Support

`resnet_predict` Function

Generate MEX Code for the `resnet_predict` Function

Call `predict` on a Test Image

Generate a Static Library for the `resnet_predict` Function

`main_resnet.cpp` File