Create a Deep Learning Experiment for Regression

This example uses:

Open Example

This example shows how to train a deep learning network for regression by using Experiment Manager. In this example, you use a regression model to predict the angles of rotation of handwritten digits. A custom metric function determines the fraction of angle predictions within an acceptable error margin from the true angles. For more information on using a regression model, see Train Convolutional Neural Network for Regression.

Open Experiment

First, open the example. Experiment Manager loads a project with a preconfigured experiment that you can inspect and run. To open the experiment, in the Experiment Browser, double-click the name of the experiment (RegressionExperiment).

An experiment definition consists of a description, a hyperparameter table, a setup function, and a collection of metric functions to evaluate the results of the experiment. For more information, see Configure Deep Learning Experiment.

The Description box contains a textual description of the experiment. For this example, the description is:

Regression model to predict angles of rotation of digits, using hyperparameters to specify:
- the number of filters used by the convolution layers
- the probability of the dropout layer in the network

The Hyperparameter Table contains the names and values of the hyperparameters used in the experiment. When you run the experiment, Experiment Manager sweeps through the hyperparameter values and trains the network multiple times. Each trial uses a different combination of the hyperparameter values specified in the table. This example uses two hyperparameters:

  • Probability sets the probability of the dropout layer in the neural network. By default, the values for this hyperparameter are specified as [0.1 0.2].

  • Filters indicates the number of filters used by the first convolution layer in the neural network. In the subsequent convolution layers, the number of filters is a multiple of this value. By default, the values of this hyperparameter are specified as [4 6 8].

The Setup Function configures the training data, network architecture, and training options for the experiment. To inspect the setup function, under Setup Function, click Edit. The setup function opens in MATLAB Editor.

In this example, the input to the setup function is a struct with fields from the hyperparameter table. The setup function returns four outputs that you use to train a network for image regression problems. The setup function has three sections.

  • Load Image Data defines the training and validation data for the experiment as 4-D arrays. The training and validation data sets each contain 5000 images of digits from 0 to 9. The regression values correspond to the angles of rotation of the digits.

  • Define Network Architecture defines the architecture for a convolutional neural network for regression.

  • Specify Training Options defines a trainingOptions object for the experiment. The example trains the network for 30 epochs. The learning rate is initially 0.001 and drops by a factor of 0.1 after 20 epochs. The software trains the network on the training data and calculates the root mean squared error (RMSE) and loss on the validation data at regular intervals during training. The validation data is not used to update the network weights.

The Metrics section specifies optional functions that evaluate the results of the experiment. Experiment Manager evaluates these functions each time it finishes training the network. To inspect a metric function, select the name of the metric function and click Edit. The metric function opens in MATLAB Editor.

This example includes a metric function Accuracy that determines the percentage of angle predictions within an acceptable error margin from the true angles. By default, the function uses a threshold of 10 degrees.

Run Experiment

In the Experiment Manager tab, click Run. Experiment Manager trains the network defined by the setup function six times. Each trial uses a different combination of hyperparameters. A table of results displays the RMSE and loss for each trial. The table also displays the accuracy of the trial, as determined by the custom metric function Accuracy.

While the experiment is running, click Training Plot to display the training plot and track the progress of each trial.

Evaluate Results

To find the best result for your experiment, sort the table of results by accuracy.

  1. Point to the Accuracy column.

  2. Click the triangle icon.

  3. Select Sort in Descending Order.

The trial with the highest accuracy appears at the top of the results table.

To test the performance of an individual trial, export the trained network and display a box plot of the residuals for each digit class.

  1. Select the trial with the highest accuracy.

  2. On the Experiment Manager tab, click Export.

  3. In the dialog window, enter the name of a workspace variable for the exported network. The default name is trainedNetwork.

  4. Create a residual box plot by calling the plotResiduals function. Use the exported network as the input to the function. For instance, in the MATLAB Command Window, enter plotResiduals(trainedNetwork).

Close Experiment

In the Experiment Browser, right-click the name of the project and select Close Project. Experiment Manager saves your results and closes all of the experiments contained in the project.

See Also

| |

Related Topics

Deep Learning Toolbox Documentation
Support