%% Adapt Code Generated in Deep Network Designer  for Use in Experiment Manager
% This example shows how to use Experiment Manager to tune the hyperparameters of a network trained in Deep Network Designer.
% You can use Deep Network Designer to create a network, import data, and train the network. You can then use Experiment Manager to sweep through a range of hyperparameter values to find optimal training options.
%% Generate Training Script
% To generate a live script to recreate the building and training of a network you construct in Deep Network Designer, on the Training tab, select Export > Generate Code for Training. Select a MAT file location and click OK. For an example showing how to train a classification network in Deep Network Designer, see Create Simple Image Classification Network Using Deep Network Designer.
%
% Deep Network Designer creates a live script and a MAT file containing the initial parameters (weights and biases) from your network. If you import data from the workspace into Deep Network Designer, then the generated MAT file contains the data as well.
% Running the generated script builds the network (including the learnable parameters from the MAT file), imports the data, sets the training options, and trains the network.
%
%% Open Experiment Manager
% Experiment Manager enables you to create deep learning experiments to train networks under various initial conditions and compare the results. You can use Experiment Manager to tune a network you initially train in Deep Network Designer.
% Open Experiment Manager.

experimentManager

% Pause on Project and click Create. Experiment Manager provides several templates that support many deep learning workflows, including image classification, image regression, sequence classification, semantic segmentation, and custom training loops.
%
% Pause on Built-In Training and click ADD.
%
% Specify the name and location for the new project and click Save. Experiment Manager opens a new experiment in the project. The Experiment pane displays the description, hyperparameters, setup function, and metrics that define the experiment.
%% Add Hyperparameters
% In the hyperparameter table, specify the values of the hyperparameters to use in the experiment. When you run the experiment, Experiment Manager trains the network using every combination of the hyperparameter values specified in the table. For this example, sweep over the initial learning rate.
% Under Hyperparameters, click Add to add a new hyperparameter to sweep over.
%
% Add the hyperparameter myInitialLearnRate. Set the hyperparameter to sweep the sequence of values 0.001:0.002:0.015.
%
%% Create Setup Function Using Generated Script
% When you create an experiment, Experiment Manager creates a setup function template. To edit this function, under Setup Function, click Edit.
%
% The empty setup function Experiment1_setup1 opens in MATLAB Editor. Experiment Manager uses the outputs of this function to call the trainNetwork function.
%
% The setup function is where you specify the training data, network architecture, and training options for the experiment.
% Copy and paste the live script generated by Deep Network Designer inside the setup function.
%
%% Adapt Setup Function Input Arguments
% Adapt the script for use in Experiment Manager by changing the function input arguments to match the variable names in the generated script. The input arguments to Experiment1_setup1 must match those the generated script uses in the call to trainNetwork.
% In the script generated by Deep Network Designer, find the variable names of the data, network, and training options at the bottom of the generated live script in the call to trainNetwork. Change the setup function input arguments to match. For example, if your generated live script calls trainNetwork with data imdsTrain, network lgraph, and training options opts, then you must make the following changes in the experiment setup function input arguments:
% Change trainingData to imdsTrain.
% Change layers to lgraph.
% Change options to opts.
%
% You can check to see if your input arguments need changing by looking for the yellow underline in the setup function input arguments.
%% Adapt Training Options
% Change the training options so that Experiment Manager conducts a hyperparameter sweep of the learning rate.
% Set the initial learning rate to params.myInitialLearnRate.
% Optionally, hide the output information by adding the additional name-value argument "Verbose",false.
%
%% Remove Call to trainNetwork
% Experiment Manager uses the outputs of the setup function to call the trainNetwork function. Remove the call to trainNetwork from the copied and pasted generated code.
% The setup function is now ready. Click Save to save your edited setup function.
%% Run Experiment
% In Experiment Manager, run the experiment by clicking Run. When you run the experiment, Experiment Manager trains the network defined by the setup function. Each trial uses one of the learning rates specified in the hyperparameter table.
% While the experiment is running, click Training Plot to display the training plot and track the progress of each trial.
% A table of results displays the accuracy and loss for each trial. When the experiment finishes, you can sort the trials by the accuracy or loss metrics to see which trial performs the best. In this experiment, Trial 6, with an initial learning rate of 0.0110, has the highest validation accuracy.
%
% To close the experiment, in the Experiment Browser pane, right-click the name of the project and select Close Project. Experiment Manager closes all of the experiments and results contained in the project.
% Copyright 2020 The MathWorks, Inc.