%% Try Multiple Pretrained Networks for Transfer Learning
% This example shows how to configure an experiment that replaces layers of
% different pretrained networks for transfer learning. To compare the
% performance of different pretrained networks for your task, edit this
% experiment and specify which pretrained networks to use. Before running
% the experiment, use functions such as
% |<docid:nnet_ref#mw_7cbd0577-4371-4eb3-829c-a9447220d89d googlenet>| to
% get links to download pretrained networks from the Add-On Explorer.
%
% Transfer learning is commonly used in deep learning applications. You can
% take a pretrained network and use it as a starting point to learn a new
% task. Fine-tuning a network with transfer learning is usually much faster
% and easier than training a network with randomly initialized weights from
% scratch. You can quickly transfer learned features to a new task using a
% smaller number of training images. 
%
% There are many pretrained networks available in Deep Learning
% Toolbox(TM). These pretrained networks have different characteristics
% that matter when choosing a network to apply to your problem. The most
% important characteristics are network accuracy, speed, and size. Choosing
% a network is generally a tradeoff between these characteristics. For more
% information, see <docid:nnet_ug#bvf9ych-1 Pretrained Deep Neural
% Networks>.

% Copyright 2019-2020 The MathWorks, Inc.

%% Open Experiment
% First, open the example.
% *<docid:nnet_ref#mw_11ce840f-eed7-4975-9c28-baeebd79f092 Experiment
% Manager>* loads a project with a preconfigured experiment that you can
% inspect and run. To open the experiment, in the *Experiment Browser*
% pane, double-click the name of the experiment
% (|TransferLearningExperiment|).
%
% <<../exp_mgr_transfer_learning_config.png>>

% Project Name: 'FlowerTransferLearningProject'

%%
% An experiment definition consists of a description, a table of
% hyperparameters, a setup function, and (optionally) a collection of
% metric functions to evaluate the results of the experiment. For more
% information, see <docid:nnet_ref#mw_92d8f99c-f283-4bc5-a5d7-7d779c4831f7
% Configure Deep Learning Experiment>.
%
% The *Description* box contains a textual description of the experiment.
% For this example, the description is:
%
%  Perform transfer learning by replacing layers in a pretrained network.
%
% The *Hyperparameters* section specifies the strategy (|Exhaustive Sweep|)
% and hyperparameter values to use for the experiment. When you run the
% experiment, Experiment Manager trains the network using every combination
% of hyperparameter values specified in the hyperparameter table. In this
% example, the hyperparameter |NetworkName| specifies the network to train
% and the value of the training option |'miniBatchSize'|.
%
% The *Setup Function* configures the training data, network architecture,
% and training options for the experiment. To inspect the setup function,
% under *Setup Function*, click *Edit*. The setup function opens in
% MATLAB(R) Editor.
%
% In this example, the setup function:
%
% * Downloads and extracts the Flowers data set, which is about 218 MB. For
% more information on this data set, see
% <docid:nnet_ug#mw_d3130147-29e6-47eb-b6f3-9a11a4c8b659 Image Data Sets>.
% * Loads a pretrained network corresponding to the hyperparameter
% |NetworkName|. The auxiliary function |findLayersToReplace| determines
% the layers in the network architecture to replace for transfer learning.
% For more information on the available pretrained networks, see
% <docid:nnet_ug#bvf9ych-1 Pretrained Deep Neural Networks>.
% * Defines a |<docid:nnet_ref#bu59f0q trainingOptions>| object for the
% experiment. The example trains the network for 10 epochs, using an
% initial learning rate of 0.0003 and validating the network every 5
% epochs.
%
% The *Metrics* section specifies optional functions that evaluate the
% results of the experiment. This example does not include any custom
% metric functions.

%% Run Experiment
% When you run the experiment, Experiment Manager trains the network
% defined by the setup function six times. Each trial uses a different
% combination of hyperparameter values. By default, Experiment Manager runs
% one trial at a time. If you have Parallel Computing Toolbox(TM), you can
% run multiple trials at the same time. For best results, before you run
% your experiment, start a parallel pool with as many workers as GPUs. For
% more information, see
% <docid:nnet_ug#mw_4858679a-7447-4736-9c57-7b8dc706648d Use Experiment
% Manager to Train Networks in Parallel>.
%
% * To run one trial of the experiment at a time, in the Experiment Manager
% toolstrip, click *Run*.
% * To run multiple trials at the same time, click *Use Parallel* and then
% *Run*. If there is no current parallel pool, Experiment Manager starts
% one using the default cluster profile. Experiment Manager then executes
% multiple simultaneous trials, depending on the number of parallel workers
% available.
%
% A table of results displays the accuracy and loss for each trial.
% 
% While the experiment is running, click *Training Plot* to display the
% training plot and track the progress of each trial.
% 
% Click *Confusion Matrix* to display the confusion matrix for the
% validation data in each completed trial.
%
% When the experiment finishes, you can sort the results table by column or
% filter trials by using the *Filters* pane. For more information, see
% <docid:nnet_ref#mw_ffdb175d-cd27-4954-8fa6-e006dbb6dfd0 Sort and Filter
% Experiment Results>.
%
% To test the performance of an individual trial, export the trained
% network or the training information for the trial. On the *Experiment
% Manager* toolstrip, select *Export* > *Trained Network* or *Export* >
% *Training Information*, respectively. For more information, see
% <docid:nnet_ref#bu6sn4c-trainedNet Trained Network> and
% <docid:nnet_ref#bu6sn4c-traininfo Training Information>.

%% Close Experiment
% In the *Experiment Browser* pane, right-click the name of the project and
% select *Close Project*. Experiment Manager closes all of the experiments
% and results contained in the project.