%% Code Generation For Object Detection Using YOLO v3 Deep Learning
% This example shows how to generate CUDA® MEX for a you only look once (YOLO) v3 object detector with custom layers. The example uses YOLO v3 object detection to illustrate:
% CUDA code generation for a deep learning network with custom layers.
% Convert a deep learning dlnetwork object into a DAGNetwork object for code generation.
% YOLO v3 improves upon YOLO v2 by adding detection at multiple scales to help detect smaller objects. Moreover, the loss function used for training is separated into mean squared error for bounding box regression and binary cross-entropy for object classification to help improve detection accuracy. This example uses the YOLO v3 network trained in the Object Detection Using YOLO v3 Deep Learning example from the Computer Vision Toolbox (TM). For more information, see Object Detection Using YOLO v3 Deep Learning.
%
% Table of Contents
% Third-Party Prerequisites
%     Required
%     Optional
% Verify GPU Environment
% YOLO v3 Network
% Pretrained YOLO v3 Network
% Prepare Pretrained Model for Code Generation
% Custom Upsample Layer
% Assemble layerGraph into a DAGNetwork
% The yolov3Detect Entry-Point Function
% Evaluate the Entry-Point Function for Object Detection
% Generate CUDA MEX
% Run the Generated MEX
% Utility Functions
% References
%
%% Third-Party Prerequisites
%% Required
% CUDA enabled NVIDIA® GPU and compatible driver.
%% Optional
% For non-MEX builds such as static, dynamic libraries or executables, this example has the following additional requirements.
% NVIDIA CUDA toolkit.
% NVIDIA cuDNN library.
% Environment variables for the compilers and libraries. For more information, see Third-party Products and Setting Up the Prerequisite Products.
%
%% Verify GPU Environment
% To verify that the compilers and libraries for running this example are set up correctly, use the coder.checkGpuInstall function.

envCfg = coder.gpuEnvConfig('host');
envCfg.DeepLibTarget = 'cudnn';
envCfg.DeepCodegen = 1;
envCfg.Quiet = 1;
coder.checkGpuInstall(envCfg);

%
%% YOLO v3 Network
% The YOLO v3 network in this example is based on SqueezeNet, and uses the feature extraction network in SqueezeNet with the addition of two detection heads at the end. The second detection head is twice the size of the first detection head, so it is better able to detect small objects. Note that any number of detection heads of different sizes can be specified based on the size of the objects to be detected. The YOLO v3 network uses anchor boxes estimated using training data to have better initial priors corresponding to the type of data set and to help the network learn to predict the boxes accurately. For information about anchor boxes, see Anchor Boxes for Object Detection.
% The YOLO v3 network in this example is illustrated in the following diagram.
%
% Each detection head predicts the bounding box coordinates (x, y, width, height), object confidence, and class probabilities for the respective anchor box masks. Therefore, for each detection head, the number of output filters in the last convolution layer is the number of anchor box mask times the number of prediction elements per anchor box. The detection heads comprise the output layer of the network.
%
%% Pretrained YOLO v3 Network
% Download the YOLO v3 network trained in the Object Detection Using YOLO v3 Deep Learning example. To train the network yourself, see Object Detection Using YOLO v3 Deep Learning.

pretrained = load("yolov3SqueezeNetVehicleExample_20a.mat");
net = pretrained.net;

%
%% Prepare Pretrained Model for Code Generation
% The pretrained YOLO v3 model is a dlnetwork object that is not supported for code generation. Extract the layer graph of the dlnetwork object by using the layerGraph function.

lGraph = layerGraph(net);

% The layer graph output by the layerGraph function does not include output layers. Add a regression layer to the layer graph for each of its outputs by using the addLayers and connectLayers functions.

outLayerIdx = 1:numel(lGraph.Layers);
isOutLayer = arrayfun(@(x) any(strcmp(x.Name, net.OutputNames)), lGraph.Layers);
outLayerIdx(~isOutLayer) = [];

for iOut = 1:numel(outLayerIdx)
    outLayer = lGraph.Layers(outLayerIdx(iOut));
    newLayer = regressionLayer('Name', [outLayer.Name '_output_' num2str(iOut)]);
    lGraph = addLayers(lGraph, newLayer);
    lGraph = connectLayers(lGraph, outLayer.Name, newLayer.Name);
end

% The number of output layers of the network is same as the number of detection heads in the network.
%
%% Custom Upsample Layer
% YOLO v3 network uses a custom upsample layer for upsampling the input image data. Here, we use a code generation compatible version of upsampleLayer mentioned in the Object Detection Using YOLO v3 Deep Learning example. The upsampleBy2Layer upsamples the input image by replicating the neighboring pixel values by a factor of 2  using the repelem function. Because code generation for the repelem function is supported only for vector or 2-D matrix inputs, the upsampleBy2Layer implementation needs to be parameterized by using the coder.target function. When running a simulation on MATLAB, a single call to repelem is sufficient as the input can be an N-D matrix. For code generation, the example uses nested for-loops so that the input to repelem function is 2-D matrix for each call to repelem in the generated code. Furthermore, for code generation layer output size is required to be constant at code generation time. Hence, UpsampleFactor property of the layer is modified to be Constant.

type('upsampleBy2Layer.m')

% Replace the upsampleLayer present in the pretrained network with its code generation compatible version upsampleBy2Layer by using the replaceLayer function.

isUpsampleLayer = arrayfun(@(x) isa(x, 'upsampleLayer'), lGraph.Layers);
layerNameToReplace = lGraph.Layers(isUpsampleLayer).Name;
lGraph = replaceLayer(lGraph, layerNameToReplace, upsampleBy2Layer(layerNameToReplace));

%
%% Assemble layerGraph into a DAGNetwork
% Assemble the layerGraph into a DAGNetwork object ready to use for code generation by using the assembleNetwork function.

dagNet = assembleNetwork(lGraph)

% Save the network to a MAT-file.

matFile = 'yolov3DAGNetwork.mat';
save(matFile, 'dagNet');

%
%% The yolov3Detect Entry-Point Function
% The yolov3Detect entry-point function takes an input image and passes it to a trained network for prediction through the yolov3Predict function. The yolov3Predict function loads the network object from the MAT-file into a persistent variable and reuses the persistent object for subsequent prediction calls. Specifically, the function uses the DAGNetwork representation of the network trained in the Object Detection Using YOLO v3 Deep Learning example. The predictions from the YOLO v3 grid cell coordinates obtained from the yolov3Predict calls are then converted to bounding box coordinates by using the supporting functions generateTiledAnchors and applyAnchorBoxOffsets.

type('yolov3Detect.m')

%
%% Evaluate the Entry-Point Function for Object Detection
% Follow these steps to evaluate the entry-point function on an image from the test data.
% Specify the confidence threshold as 0.5 to keep only detections with confidence scores above this value.
% Specify the overlap threshold as 0.5 to remove overlapping detections.
% Read an image from the input data.
% Use the entry-point function yolov3Detect to get the predicted bounding boxes, confidence scores, and class labels.
% Display the image with bounding boxes and confidence scores.
% Define the desired thresholds.

confidenceThreshold = 0.5;
overlapThreshold = 0.5;

% Get the network input size from the input layer of the trained network and the number of network outputs.

networkInputIdx = arrayfun(@(x)isa(x,'nnet.cnn.layer.ImageInputLayer'),net.Layers);
networkInputSize = net.Layers(networkInputIdx).InputSize;
networkOutputs = numel(dagNet.OutputNames);

% Read the example image data obtained from the labeled data set from the Object Detection Using YOLO v3 Deep Learning example. This image contains one instance of an object of type vehicle.

I = imread('vehicleImage.jpg');

% Specify the class names.

classNames = {'vehicle'};

% Invoke the detect method on YOLO v3 network and display the results.

[bboxes,scores,~] = yolov3Detect(matFile, I, networkInputSize, networkOutputs, confidenceThreshold, overlapThreshold, classNames);

% Display the detections on the image
IAnnotated = insertObjectAnnotation(I, 'rectangle', bboxes, scores);
figure
imshow(IAnnotated)

%
%% Generate CUDA MEX
% To generate CUDA® code for the yolov3Detect entry-point function, create a GPU code configuration object for a MEX target and set the target language to C++. Use the coder.DeepLearningConfig function to create a CuDNN deep learning configuration object and assign it to the DeepLearningConfig property of the GPU code configuration object.

cfg = coder.gpuConfig('mex');
cfg.TargetLang = 'C++';
cfg.DeepLearningConfig = coder.DeepLearningConfig('cudnn');

args = {coder.Constant(matFile), I, coder.Constant(networkInputSize), networkOutputs, ...
    confidenceThreshold, overlapThreshold, classNames};

codegen -config cfg yolov3Detect -args args

% To generate CUDA® code for TensorRT target create and use a TensorRT deep learning configuration object instead of the CuDNN configuration object. Similarly, to generate code for MKLDNN target, create a CPU code configuration object and use MKLDNN deep learning configuration object as its DeepLearningConfig property.
%
%% Run the Generated MEX
% Call the generated CUDA MEX with the same image input I as before and display the results.

[bboxes, scores, labels] = yolov3Detect_mex(matFile, I, networkInputSize, networkOutputs, ...
    confidenceThreshold, overlapThreshold, classNames);
figure;
IAnnotated = insertObjectAnnotation(I, 'rectangle', bboxes, scores);
imshow(IAnnotated);

%
%% Utility Functions
% The utillity functions listed below are similar to the ones used in Object Detection Using YOLO v3 Deep Learning example, except for the preProcessData function. In this example, we only pre-process the image data unlike in Object Detection Using YOLO v3 Deep Learning example, where the bounding boxes are also processed.

type('applyActivations.m')

%

type('extractPredictions.m')

%

type('generateTiledAnchors.m')

%

type('applyAnchorBoxOffsets.m')

%

type('preprocessData.m')

%% References
% 1. Redmon, Joseph, and Ali Farhadi. “YOLOv3: An Incremental Improvement.” Preprint, submitted April 8, 2018. https://arxiv.org/abs/1804.02767.
% Copyright 2020 The MathWorks, Inc.