Smile Detection by Using OpenCV Code in Simulink

This example shows how to build a smile detector by using the OpenCV Importer. The detector estimates the intensity of the smile on a face image. Based on the estimated intensity, the detector identifies an appropriate emoji from its database, and then places the emoji on the smiling face.

In this example, you first import an OpenCV function into Simulink by using OpenCV Code Import Wizard. The wizard creates a Simulink^® library that contains a subsystem and a C Caller block for the specified OpenCV function. The subsystem is then used in a preconfigured Simulink model to accept the face image for smile detection. You can generate C++ code from the model, and then deploy the code on your target hardware.

In this example, you learn how to:

Import an OpenCV function into a Simulink library.
Use blocks from a generated library in a Simulink model.
Generate C++ code from a Simulink model.
Deploy the model on the Raspberry Pi hardware.

Required Products

Computer Vision Toolbox™ OpenCV Interface for Simulink
Computer Vision Toolbox
Embedded Coder^®

Set Up Your C++ Compiler

To build the OpenCV libraries, identify a compatible C++ compiler for your operating system, as described in Compiler used to build OpenCV libraries. Configure the identified compiler by using the mex -setup c++ command. For more information, see Choose a C++ Compiler.

Model Description

In this example, a smile detector is implemented by using the Simulink model smileDetect.slx.

In this model, the subsystem_slwrap_detectAndDraw subsystem resides in the Smile_Detect_OpenCVCallerLib library. You create the subsystem_slwrap_detectAndDraw subsystem by using the OpenCV Importer. The subsystem accepts a face image from the Image From Workspace block and provides these output values.

Output Port	Description
Out1	Face image with a circle
Out2	Intensity of the smile
Out3	x coordinate of center of the circle
Out4	y coordinate of center of the circle
Out5	Radius of the circle

The MATLAB Function block accepts four emoji images from Image from File blocks. The smile intensity of the emoji in these images ranges from low to high. From the four images, the block identifies the most appropriate emoji for the estimated intensity and places it on the face image. The output is then provided to the Video Viewer blocks.

Step 1: Import OpenCV Function to Create a Simulink Library

To start the OpenCV Importer app, click Apps on the MATLAB^® Toolstrip. In the Welcome page, specify the Project name as Smile_Detector. Make sure that the project name does not contain any spaces. Click Next.
In Specify OpenCV Library, specify these file locations and click Next.
- Project root folder: Specify the path to the example folder asmatlab\toolbox\simulink\supportpackages\simulinkopencv\example\SmileDetector. This folder contains all the supporting files required to run this example. All your output files are saved to this folder.
- Source files: Specify the path of the .cpp file located inside your project folder as smiledetect.cpp.
- Include files: Specify the path of the .hpp header file located inside your project folder as smiledetect.hpp.
Analyze your library to find the functions and types for import. Once the analysis is completed, click Next. Select the detectAndDraw function and click Next.
From What to import, select the I/O Type for img as Input, and then click Next.
In Create Simulink Library, verify the default values and click Next.
A Simulink library Smile_Detector_OpenCVCallerLib is created from your OpenCV code into the project root folder. The library contains a subsystem and a C Caller block. You can use any of these blocks for model simulation. In this example, the subsystem subsystem_slwrap_detectAndDraw is used.

Step 2: Use Generated Subsystem in Simulink Model

To use the generated subsystem subsystem_slwrap_detectAndDraw with the Simulink model smileDetect.slx:

In your MATLAB Current Folder, right-click the model smileDetect.slx and click Open from the context menu. In the model, delete the existing subsystem_slwrap_detectAndDraw subsystem and drag the generated subsystem subsystem_slwrap_detectAndDraw from the Smile_Detector_OpenCVCallerLib library to the model. Connect the subsystem to the MATLAB Function block.

Double-click the subsystem and specify these parameter values.

Parameters	Values	Description
Rows	512	Number of rows in the output image
Columns	512	Number of columns in the output image
Channels	3	Number of channels in the output image
Underlying Type	uint8	Underlying data type of OpenCV `Mat`
is Image	on	Whether input is an image or a matrix

Click Apply, and then click OK.

Step 3: Simulate the Smile Detector

On the Simulink Toolstrip, in the Simulation tab, click to simulate the model. After the simulation is complete, the Video Viewer block displays an image with an emoji on the face. The emoji represents the intensity of the smile.

Step 4: Generate C++ Code from the Smile Detector Model

Before you generate the code from the model, you must first ensure that you have write permission in your current folder.

To generate C++ code:

Open the smileDetect_codegen.slx model from your MATLAB Current Folder.
To review the model settings:
On the Apps tab of the Simulink toolstrip, select Embedded Coder. The C++ Code tab appears.
In the Settings drop-down list, click C/C++ Code generation settings to open the Configuration Parameters and verify these settings.
- In the Code Generation pane, under Target selection, Language is set to C++.
- In the Interface under Code Generation, Array layout in the Data exchange interface category is set to Row-major.
Connect the generated subsystem subsystem_slwrap_detectAndDraw to the MATLAB Function block.
To generate C++ code, under the C++ Code tab, click the Generate Code drop-down list and then click Build. After the model finishes generating code, the Code Generation Report opens. You can inspect the generated code. The build process creates a zip file called smileDetect_with_ToOpenCV.zip in your current MATLAB working folder.

Deploy the Smile Detector on the Raspberry Pi Hardware

Before you deploy the model, connect the Raspberry Pi to your computer. Wait until the PWR LED on the hardware starts blinking.

In the Settings drop-down list, click Hardware Implementation to open the Configuration Parameters and verify these settings:

Set the Hardware board to Raspberry Pi. The Device Vendor is automatically set to ARM Compatible.
In the Code Generation pane, under Target selection, Language is set to C++. Under Build process, Zip file name is set to smileDetect_with_ToOpenCV.zip. Under Toolchain settings, the Toolchain is specified as GNU GCC Raspberry Pi.

To deploy the code to your Raspberry Pi hardware:

From the generated zip file, copy these files to your Raspberry Pi hardware.
- smiledetect.zip
- smileDetect.mk
- main.cpp
In Raspberry Pi, go to the location where you saved the files and enter this command. This command generates an elf file.
```
make -f smileDetect.mk
```
Run the executable on Raspberry Pi. On successful execution, you see the output on Raspberry Pi with an emoji placed on the face image.
```
smileDetect.elf
```

Documentation