This example shows how to quantize learnable parameters in the
convolution layers of a neural network, and explore the behavior of the quantized network. In
this example, you quantize the LogoNet neural network. Quantization helps
reduce the memory requirement of a deep neural network by quantizing weights, biases and
activations of network layers to 8-bit scaled integer data types. Use MATLAB® to retrieve the
prediction results from the target device.
To run this example, you need the products listed under FPGA in
Quantization Workflow Prerequisites.
Create a file in your current working directory called
getLogoNetwork.m. Enter these lines into the file:
function net = getLogoNetwork() data = getLogoData(); net = data.convnet; end function data = getLogoData() if ~isfile('LogoNet.mat') url = 'https://www.mathworks.com/supportfiles/gpucoder/cnn_models/logo_detection/LogoNet.mat'; websave('LogoNet.mat',url); end data = load('LogoNet.mat'); end
snet = getLogoNetwork();
snet =
SeriesNetwork with properties:
Layers: [22×1 nnet.cnn.layer.Layer]
InputNames: {'imageinput'}
OutputNames: {'classoutput'}The calibration data is used to collect the dynamic ranges of the weights and biases in the convolution and fully connected layers of the network and the dynamic ranges of the activations in all layers of the network. For the best quantization results, the calibration data must be representative of inputs to the network.
The validation data is used to test the network after quantization to understand the effects of the limited range and precision of the quantized convolution layers in the network.
In this example, use the images in the logos_dataset data set. Define
an augmentedImageDatastore object to resize the data for the network. Then,
split the data into calibration and validation data sets.
curDir = pwd; newDir = fullfile(matlabroot,'examples','deeplearning_shared','data','logos_dataset.zip'); copyfile(newDir,curDir); unzip('logos_dataset.zip'); imageData = imageDatastore(fullfile(curDir,'logos_dataset'),... 'IncludeSubfolders',true,'FileExtensions','.JPG','LabelSource','foldernames'); [calibrationData, validationData] = splitEachLabel(imageData, 0.5,'randomized');
Create a dlquantizer object and specify the network to quantize.
dlQuantObj = dlquantizer(snet,'ExecutionEnvironment','FPGA');
Use the calibrate function to exercise the network with sample inputs
and collect range information. The calibrate function exercises the network
and collects the dynamic ranges of the weights and biases in the convolution and fully
connected layers of the network and the dynamic ranges of the activations in all layers of
the network. The function returns a table. Each row of the table contains range information
for a learnable parameter of the optimized network.
dlQuantObj.calibrate(calibrationData)
ans =
Optimized Layer Name Network Layer Name Learnables / Activations MinValue MaxValue
____________________________ __________________ ________________________ ___________ __________
{'conv_1_Weights' } {'conv_1' } "Weights" -0.048978 0.039352
{'conv_1_Bias' } {'conv_1' } "Bias" 0.99996 1.0028
{'conv_2_Weights' } {'conv_2' } "Weights" -0.055518 0.061901
{'conv_2_Bias' } {'conv_2' } "Bias" -0.00061171 0.00227
{'conv_3_Weights' } {'conv_3' } "Weights" -0.045942 0.046927
{'conv_3_Bias' } {'conv_3' } "Bias" -0.0013998 0.0015218
{'conv_4_Weights' } {'conv_4' } "Weights" -0.045967 0.051
{'conv_4_Bias' } {'conv_4' } "Bias" -0.00164 0.0037892
{'fc_1_Weights' } {'fc_1' } "Weights" -0.051394 0.054344
{'fc_1_Bias' } {'fc_1' } "Bias" -0.00052319 0.00084454
{'fc_2_Weights' } {'fc_2' } "Weights" -0.05016 0.051557
{'fc_2_Bias' } {'fc_2' } "Bias" -0.0017564 0.0018502
{'fc_3_Weights' } {'fc_3' } "Weights" -0.050706 0.04678
{'fc_3_Bias' } {'fc_3' } "Bias" -0.02951 0.024855
{'imageinput' } {'imageinput'} "Activations" 0 255
{'imageinput_normalization'} {'imageinput'} "Activations" -139.34 198.72
Create a target object with a custom name for your target device and an interface to connect your target device to the host computer. Interface options are JTAG and Ethernet. To create the target object, enter:
hTarget = dlhdl.Target('Intel', 'Interface', 'JTAG');
Define a metric function to use to compare the behavior of the network before and after quantization. Save this function in a local file.
function accuracy = hComputeAccuracy(predictionScores, net, dataStore) %% hComputeAccuracy test helper function computes model level accuracy statistics % Copyright 2020 The MathWorks, Inc. % Load ground truth groundTruth = dataStore.Labels; % Compare with predicted label with actual ground truth predictionError = {}; for idx=1:numel(groundTruth) [~, idy] = max(predictionScores(idx, :)); yActual = net.Layers(end).Classes(idy); predictionError{end+1} = (yActual == groundTruth(idx)); %#ok end % Sum all prediction errors. predictionError = [predictionError{:}]; accuracy = sum(predictionError)/numel(predictionError); end
dlQuantizationOptions ObjectSpecify the metric function in a dlquantizationOptions object.
options = dlquantizationOptions('MetricFcn', ... {@(x)hComputeModelAccuracy(x, snet, validationData)},'Bitstream','arria10soc_int8',... 'Target',hTarget);
To compile and deploy the quantized network, run the validate
function of the dlquantizer object. Use the validate
function to quantize the learnable parameters in the convolution layers of the network and
exercise the network. This function uses the output of the compile function to program the
FPGA board by using the programming file. It also downloads the network weights and biases.
The deploy function checks for the Intel Quartus tool and the supported tool version. It
then starts programming the FPGA device by using the sof file, displays progress messages,
and the time it takes to deploy the network. The function uses the metric function defined
in the dlquantizationOptions object to compare the results of the network
before and after quantization.
prediction = dlQuantObj.validate(validationData,options);
offset_name offset_address allocated_space
_______________________ ______________ _________________
"InputDataOffset" "0x00000000" "48.0 MB"
"OutputResultOffset" "0x03000000" "4.0 MB"
"SystemBufferOffset" "0x03400000" "60.0 MB"
"InstructionDataOffset" "0x07000000" "8.0 MB"
"ConvWeightDataOffset" "0x07800000" "8.0 MB"
"FCWeightDataOffset" "0x08000000" "12.0 MB"
"EndOffset" "0x08c00000" "Total: 140.0 MB"
### Programming FPGA Bitstream using JTAG...
### Programming the FPGA bitstream has been completed successfully.
### Loading weights to Conv Processor.
### Conv Weights loaded. Current time is 16-Jul-2020 12:45:10
### Loading weights to FC Processor.
### FC Weights loaded. Current time is 16-Jul-2020 12:45:26
### Finished writing input activations.
### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 13570959 0.09047 30 380609145 11.8
conv_module 12667786 0.08445
conv_1 3938907 0.02626
maxpool_1 1544560 0.01030
conv_2 2910954 0.01941
maxpool_2 577524 0.00385
conv_3 2552707 0.01702
maxpool_3 676542 0.00451
conv_4 455434 0.00304
maxpool_4 11251 0.00008
fc_module 903173 0.00602
fc_1 536164 0.00357
fc_2 342643 0.00228
fc_3 24364 0.00016
* The clock frequency of the DL processor is: 150MHz
### Finished writing input activations.
### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 13570364 0.09047 30 380612682 11.8
conv_module 12667103 0.08445
conv_1 3939296 0.02626
maxpool_1 1544371 0.01030
conv_2 2910747 0.01940
maxpool_2 577654 0.00385
conv_3 2551829 0.01701
maxpool_3 676548 0.00451
conv_4 455396 0.00304
maxpool_4 11355 0.00008
fc_module 903261 0.00602
fc_1 536206 0.00357
fc_2 342688 0.00228
fc_3 24365 0.00016
* The clock frequency of the DL processor is: 150MHz
### Finished writing input activations.
### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 13571561 0.09048 30 380608338 11.8
conv_module 12668340 0.08446
conv_1 3939070 0.02626
maxpool_1 1545327 0.01030
conv_2 2911061 0.01941
maxpool_2 577557 0.00385
conv_3 2552082 0.01701
maxpool_3 676506 0.00451
conv_4 455582 0.00304
maxpool_4 11248 0.00007
fc_module 903221 0.00602
fc_1 536167 0.00357
fc_2 342643 0.00228
fc_3 24409 0.00016
* The clock frequency of the DL processor is: 150MHz
### Finished writing input activations.
### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 13569862 0.09047 30 380613327 11.8
conv_module 12666756 0.08445
conv_1 3939212 0.02626
maxpool_1 1543267 0.01029
conv_2 2911184 0.01941
maxpool_2 577275 0.00385
conv_3 2552868 0.01702
maxpool_3 676438 0.00451
conv_4 455353 0.00304
maxpool_4 11252 0.00008
fc_module 903106 0.00602
fc_1 536050 0.00357
fc_2 342645 0.00228
fc_3 24409 0.00016
* The clock frequency of the DL processor is: 150MHz
### Finished writing input activations.
### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 13570823 0.09047 30 380619836 11.8
conv_module 12667607 0.08445
conv_1 3939074 0.02626
maxpool_1 1544519 0.01030
conv_2 2910636 0.01940
maxpool_2 577769 0.00385
conv_3 2551800 0.01701
maxpool_3 676795 0.00451
conv_4 455859 0.00304
maxpool_4 11248 0.00007
fc_module 903216 0.00602
fc_1 536165 0.00357
fc_2 342643 0.00228
fc_3 24406 0.00016
* The clock frequency of the DL processor is: 150MHz
offset_name offset_address allocated_space
_______________________ ______________ _________________
"InputDataOffset" "0x00000000" "48.0 MB"
"OutputResultOffset" "0x03000000" "4.0 MB"
"SystemBufferOffset" "0x03400000" "60.0 MB"
"InstructionDataOffset" "0x07000000" "8.0 MB"
"ConvWeightDataOffset" "0x07800000" "8.0 MB"
"FCWeightDataOffset" "0x08000000" "12.0 MB"
"EndOffset" "0x08c00000" "Total: 140.0 MB"
### FPGA bitstream programming has been skipped as the same bitstream is already loaded on the target FPGA.
### Deep learning network programming has been skipped as the same network is already loaded on the target FPGA.
### Finished writing input activations.
### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 13572329 0.09048 10 127265075 11.8
conv_module 12669135 0.08446
conv_1 3939559 0.02626
maxpool_1 1545378 0.01030
conv_2 2911243 0.01941
maxpool_2 577422 0.00385
conv_3 2552064 0.01701
maxpool_3 676678 0.00451
conv_4 455657 0.00304
maxpool_4 11227 0.00007
fc_module 903194 0.00602
fc_1 536140 0.00357
fc_2 342688 0.00228
fc_3 24364 0.00016
* The clock frequency of the DL processor is: 150MHz
### Finished writing input activations.
### Running single input activations.
Deep Learning Processor Profiler Performance Results
LastLayerLatency(cycles) LastLayerLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 13572527 0.09048 10 127266427 11.8
conv_module 12669266 0.08446
conv_1 3939776 0.02627
maxpool_1 1545632 0.01030
conv_2 2911169 0.01941
maxpool_2 577592 0.00385
conv_3 2551613 0.01701
maxpool_3 676811 0.00451
conv_4 455418 0.00304
maxpool_4 11348 0.00008
fc_module 903261 0.00602
fc_1 536205 0.00357
fc_2 342689 0.00228
fc_3 24365 0.00016
* The clock frequency of the DL processor is: 150MHzExamine the MetricResults.Result field of the validation output to
see the performance of the quantized network.
prediction.MetricResults.Result
ans =
NetworkImplementation MetricOutput
_____________________ ____________
{'Floating-Point'} 0.9875
{'Quantized' } 0.9875
Examine the QuantizedNetworkFPS field of the validation output to see
the frames per second performance of the quantized network.
prediction.QuantizedNetworkFPS
ans = 11.8126