Try Multiple Pretrained Networks for Transfer Learning
This example shows how to configure an experiment that replaces layers of different pretrained networks for transfer learning. To compare the performance of different pretrained networks for your task, edit this experiment and specify which pretrained networks to use. Before running the experiment, use functions such as googlenet
Transfer learning is commonly used in deep learning applications. You can take a pretrained network and use it as a starting point to learn a new task. Fine-tuning a network with transfer learning is usually much faster and easier than training a network with randomly initialized weights from scratch. You can quickly transfer learned features to a new task using a smaller number of training images.
There are many pretrained networks available in Deep Learning Toolbox™. These pretrained networks have different characteristics that matter when choosing a network to apply to your problem. The most important characteristics are network accuracy, speed, and size. Choosing a network is generally a tradeoff between these characteristics. For more information, see Pretrained Deep Neural Networks.
Open Experiment
First, open the example. Experiment Manager loads a project with a preconfigured experiment that you can inspect and run. To open the experiment, in the Experiment Browser pane, double-click the name of the experiment (TransferLearningExperiment).
An experiment definition consists of a description, a table of hyperparameters, a setup function, and (optionally) a collection of metric functions to evaluate the results of the experiment. For more information, see Configure Deep Learning Experiment.
The Description box contains a textual description of the experiment. For this example, the description is:
Perform transfer learning by replacing layers in a pretrained network.
The Hyperparameters section specifies the strategy (Exhaustive Sweep) and hyperparameter values to use for the experiment. When you run the experiment, Experiment Manager trains the network using every combination of hyperparameter values specified in the hyperparameter table. In this example, the hyperparameter NetworkName specifies the network to train and the value of the training option 'miniBatchSize'.
The Setup Function configures the training data, network architecture, and training options for the experiment. To inspect the setup function, under Setup Function, click Edit. The setup function opens in MATLAB® Editor.
In this example, the setup function:
Downloads and extracts the Flowers data set, which is about 218 MB. For more information on this data set, see Image Data Sets.
Loads a pretrained network corresponding to the hyperparameter
NetworkName. The auxiliary functionfindLayersToReplacedetermines the layers in the network architecture to replace for transfer learning. For more information on the available pretrained networks, see Pretrained Deep Neural Networks.Defines a
trainingOptions
The Metrics section specifies optional functions that evaluate the results of the experiment. This example does not include any custom metric functions.
Run Experiment
When you run the experiment, Experiment Manager trains the network defined by the setup function six times. Each trial uses a different combination of hyperparameter values. By default, Experiment Manager runs one trial at a time. If you have Parallel Computing Toolbox™, you can run multiple trials at the same time. For best results, before you run your experiment, start a parallel pool with as many workers as GPUs. For more information, see Use Experiment Manager to Train Networks in Parallel.
To run one trial of the experiment at a time, in the Experiment Manager toolstrip, click Run.
To run multiple trials at the same time, click Use Parallel and then Run. If there is no current parallel pool, Experiment Manager starts one using the default cluster profile. Experiment Manager then executes multiple simultaneous trials, depending on the number of parallel workers available.
A table of results displays the accuracy and loss for each trial.
While the experiment is running, click Training Plot to display the training plot and track the progress of each trial.
Click Confusion Matrix to display the confusion matrix for the validation data in each completed trial.
When the experiment finishes, you can sort the results table by column or filter trials by using the Filters pane. For more information, see Sort and Filter Experiment Results.
To test the performance of an individual trial, export the trained network or the training information for the trial. On the Experiment Manager toolstrip, select Export > Trained Network or Export > Training Information, respectively. For more information, see net and info.
Close Experiment
In the Experiment Browser pane, right-click the name of the project and select Close Project. Experiment Manager closes all of the experiments and results contained in the project.
See Also
Experiment Manager | googlenet | trainingOptions | trainNetwork