%% Vehicle Detection Using DAG Network Based YOLO v2 Deployed to FPGA
% This example shows how to train and deploy a you look only once (YOLO) v2 object detector.
% Deep learning is a powerful machine learning technique that you can use  to train robust object detectors. Several techniques for object  detection exist, including Faster R-CNN and you only look once (YOLO)  v2. This example trains a YOLO v2 vehicle detector using the trainYOLOv2ObjectDetector function.
%
%% Load Dataset
% This example uses a small vehicle dataset that contains 295 images. Many of these images come from the Caltech Cars 1999 and 2001 data sets, available at the Caltech Computational Vision website, created by Pietro Perona and used with permission. Each image contains one or two labeled instances of a vehicle. A small  dataset is useful for exploring the YOLO v2 training procedure, but in  practice, more labeled images are needed to train a robust detector.  Unzip the vehicle images and load the vehicle ground truth data.

unzip vehicleDatasetImages.zip
data = load('vehicleDatasetGroundTruth.mat');
vehicleDataset = data.vehicleDataset;

% The vehicle data is stored in a two-column table, where the first column contains the image file paths and the second column contains the vehicle bounding boxes.

% Add the fullpath to the local vehicle data folder.
vehicleDataset.imageFilename = fullfile(pwd,vehicleDataset.imageFilename);

% Split the dataset into training and test sets. Select 60% of the data for training and the rest for testing the trained detector.

rng(0);
shuffledIndices = randperm(height(vehicleDataset));
idx = floor(0.6 * length(shuffledIndices) );
trainingDataTbl = vehicleDataset(shuffledIndices(1:idx),:);
testDataTbl = vehicleDataset(shuffledIndices(idx+1:end),:);

% Use imageDatastore and boxLabelDataStore to create datastores for loading the image and label data during training and evaluation.

imdsTrain = imageDatastore(trainingDataTbl{:,'imageFilename'});
bldsTrain = boxLabelDatastore(trainingDataTbl(:,'vehicle'));

imdsTest = imageDatastore(testDataTbl{:,'imageFilename'});
bldsTest = boxLabelDatastore(testDataTbl(:,'vehicle'));

% Combine image and box label datastores.

trainingData = combine(imdsTrain,bldsTrain);
testData = combine(imdsTest,bldsTest);

%
%% Create a YOLO v2 Object Detection Network
% A YOLO v2 object detection network is composed of two subnetworks. A  feature extraction network followed by a detection network. The feature  extraction network is typically a pretrained CNN (for details, see Pretrained Deep Neural Networks). This example uses ResNet-18 for feature extraction. You can also use  other pretrained networks such as MobileNet v2 or ResNet-50 depending on application requirements. The detection sub-network is a small CNN compared to the feature extraction network and is composed  of a few convolutional layers and layers specific for YOLO v2.
% Use the yolov2Layers function to create a YOLO v2 object detection network automatically given a pretrained ResNet-18 feature extraction network. yolov2Layers requires you to specify several inputs that parameterize a YOLO v2 network:
% Network input size
% Anchor boxes
% Feature extraction network
% First, specify the network input size and the number of classes. When  choosing the network input size, consider the minimum size required by  the network itself, the size of the training images, and the  computational cost incurred by processing data at the selected size.  When feasible, choose a network input size that is close to the size of  the training image and larger than the input size required for the  network. To reduce the computational cost of running the example,  specify a network input size of [224 224 3], which is the minimum size  required to run the network.

inputSize = [224 224 3];

% Define the number of object classes to detect.

numClasses = width(vehicleDataset)-1;

% Note that the training images used in this example are bigger than  224-by-224 and vary in size, so you must resize the images in a  preprocessing step prior to training.
% Next, use  estimateAnchorBoxes to estimate anchor boxes based on the size of objects in the training  data. To account for the resizing of the images prior to training,  resize the training data for estimating anchor boxes. Use transform to preprocess the training data, then define the number of anchor boxes and estimate the anchor boxes. Resize the training data to the input  image size of the network using the supporting function yolo_preprocessData.

trainingDataForEstimation = transform(trainingData,@(data)yolo_preprocessData(data,inputSize));
numAnchors = 7;
[anchorBoxes, meanIoU] = estimateAnchorBoxes(trainingDataForEstimation, numAnchors)

% For more information on choosing anchor boxes, see Estimate Anchor Boxes From Training Data (Computer Vision Toolbox) (Computer Vision Toolbox™) and Anchor Boxes for Object Detection (Computer Vision Toolbox).
% Now, use resnet18 to load a pretrained ResNet-18 model.

featureExtractionNetwork = resnet18;

% Select 'res4b_relu' as the feature extraction layer to replace the layers after 'res4b_relu' with the detection subnetwork. This feature extraction layer outputs  feature maps that are downsampled by a factor of 16. This amount of  downsampling is a good trade-off between spatial resolution and the  strength of the extracted features, as features extracted further down  the network encode stronger image features at the cost of spatial  resolution. Choosing the optimal feature extraction layer requires  empirical analysis.

featureLayer = 'res4b_relu';

% Create the YOLO v2 object detection network. .

lgraph = yolov2Layers(inputSize,numClasses,anchorBoxes,featureExtractionNetwork,featureLayer);

% You can visualize the network using analyzeNetwork or Deep Network Designer from Deep Learning Toolbox™.
% If more control is required over the YOLO v2 network architecture, use  Deep Network Designer to design the YOLO v2 detection network manually.  For more information, see Design a YOLO v2 Detection Network (Computer Vision Toolbox).
%
%% Data Augmentation
% Data augmentation is used to improve network accuracy by randomly  transforming the original data during training. By using data  augmentation you can add more variety to the training data without  actually having to increase the number of labeled training samples.
% Use transform to augment the training data by randomly flipping the image and  associated box labels horizontally. Note that data augmentation is not  applied to the test and validation data. Ideally, test and validation  data should be representative of the original data and is left  unmodified for unbiased evaluation.

augmentedTrainingData = transform(trainingData,@yolo_augmentData);

%
%% Preprocess Training Data and Train YOLO v2 Object Detector
% Preprocess the augmented training data, and the validation data to prepare for training.

preprocessedTrainingData = transform(augmentedTrainingData,@(data)yolo_preprocessData(data,inputSize));

% Use trainingOptions to specify network training options. Set 'ValidationData' to the preprocessed validation data. Set 'CheckpointPath' to a temporary location. This enables the saving of partially trained  detectors during the training process. If training is interrupted, such  as by a power outage or system failure, you can resume training from the saved checkpoint.

options = trainingOptions('sgdm', ...
        'MiniBatchSize', 16, ....
        'InitialLearnRate',1e-3, ...
        'MaxEpochs',20,...
        'CheckpointPath', tempdir, ...
        'Shuffle','never');

% Use trainYOLOv2ObjectDetector function to train YOLO v2 object detector.

[detector,info] = trainYOLOv2ObjectDetector(preprocessedTrainingData,lgraph,options);

% As a quick test, run the detector on one test image. Make sure you resize the image to the same size as the training images.

I = imread(testDataTbl.imageFilename{2});
I = imresize(I,inputSize(1:2));
[bboxes,scores] = detect(detector,I);

% Display the results.

I_new = insertObjectAnnotation(I,'rectangle',bboxes,scores);
figure
imshow(I_new)

%
%% Load Pretrained Network
% Load the pretrained network.

snet=detector.Network;
I_pre=yolo_pre_proc(I);

% Use analyzeNetwork to obtain information about the network layers:

analyzeNetwork(snet)

%
%
%% Create Target Object
% Create a target object for your target device with a vendor name and an interface to connect your target device to the host computer. Interface options are JTAG (default) and Ethernet. Vendor options are Intel or Xilinx. Use the installed Xilinx Vivado Design Suite over an Ethernet connection to program the device.

hTarget = dlhdl.Target('Xilinx', 'Interface', 'Ethernet');

%
%% Create Workflow Object
% Create an object of the dlhdl.Workflow class. When you create the object, specify the network and the bitstream name. Specify the saved pre-trained series network, trainedNetNoCar, as the network. Make sure the bitstream name matches the data type and the FPGA board that you are targeting. In this example, the target FPGA board is the Zynq UltraScale+ MPSoC ZCU102 board. The bitstream uses a single data type.

hW=dlhdl.Workflow('Network', snet, 'Bitstream', 'zcu102_single','Target',hTarget)

%
%% Compile YOLO v2 Object Detector
% To compile the snet series network, run the compile function of the dlhdl.Workflow object .

dn = hW.compile

%
%% Program the Bitstream onto FPGA and Download Network Weights
% To deploy the network on the Zynq® UltraScale+™ MPSoC ZCU102 hardware, run the deploy function of the dlhdl.Workflow object . This function uses the output of the compile function to program the FPGA board by using the programming file.The function also downloads the network weights and biases. The deploy function checks for the Xilinx Vivado tool and the supported tool version. It then starts programming the FPGA device by using the bitstream, displays progress messages and the time it takes to deploy the network.

hW.deploy

%
%% Load the Example Image and Run The Prediction
% Execute the predict function on the dlhdl.Workflow object and display the result:

[prediction, speed] = hW.predict(I_pre,'Profile','on');

% Display the prediction results.

[bboxesn, scoresn, labelsn] = yolo_post_proc(prediction,I_pre,anchorBoxes,{'Vehicle'});
I_new3 = insertObjectAnnotation(I,'rectangle',bboxesn,scoresn);
figure
imshow(I_new3)

%
%
%
% Copyright 2020 The MathWorks, Inc.
%