%% Image Regression Using Bayesian Optimization
% This template function defines the training data, network architecture, and 
% training options for an image regression experiment. Experiment Manager uses 
% the outputs of this function to call the <matlab:doc('trainNetwork') |trainNetwork|> 
% function. For more information on image regression, see <matlab:helpview('deeplearning','ExperimentManagerRegressionExample') 
% Create a Deep Learning Experiment for Regression>. For more information on using 
% Bayesian optimization, see <matlab:helpview('deeplearning','ExpMgrBayesOptExample') 
% Tune Experiment Hyperparameters by Using Bayesian Optimization>. For more information 
% on built-in experiments, see <matlab:helpview('deeplearning','exp-mgr-create-experiment') 
% Configure Built-in Training Experiment>.
% 
% Modify this function as a template to set up your own experiment.
%% Input
%% 
% * |params| is a structure with fields from the Experiment Manager hyperparameter 
% table.
%% Output
%% 
% * |XTrain| is a 4-D array containing the training images.
% * |TTrain| is a 1-D array containing the regression values for training,
% * |layers| is a layer graph that defines the neural network architecture.
% * |options| is a |trainingOptions| object.
function [XTrain,TTrain,layers,options] = {functionName}(params)
%% Load Training Data
% Load the training and validation data for the experiment as 4-D arrays. The 
% training and validation data sets each contain 5000 images from the Digits data 
% set. Each image shows a digit from 0 to 9, rotated by a certain angle. The regression 
% values correspond to the angles of rotation of the digits.
[XTrain,~,TTrain] = digitTrain4DArrayData;
[XValidation,~,TValidation] = digitTest4DArrayData;
%% 
% Replace the data with your own training and validation data. For more information, 
% see <matlab:helpview('deeplearning','data-sets-for-deep-learning') Data Sets 
% for Deep Learning>.
%% Define Network Architecture
% Define the convolutional neural network architecture in the layer graph |layers|.  
layers = [
    imageInputLayer([28 28 1])
    convolution2dLayer(3,8,'Padding','same')
    batchNormalizationLayer
    reluLayer
    averagePooling2dLayer(2,'Stride',2)
    convolution2dLayer(3,16,'Padding','same')
    batchNormalizationLayer
    reluLayer
    averagePooling2dLayer(2,'Stride',2)
    convolution2dLayer(3,32,'Padding','same')
    batchNormalizationLayer
    reluLayer
    convolution2dLayer(3,32,'Padding','same')
    batchNormalizationLayer
    reluLayer
    dropoutLayer(0.2)
    fullyConnectedLayer(1)
    regressionLayer];
%% 
% Replace the layer graph with your own layer graph to train. For more information, 
% see <matlab:helpview('deeplearning','list_of_layers') List of Deep Learning 
% Layers>.
%% Specify Training Options
% Call the <matlab:doc('trainingOptions') |trainingOptions|> function to specify 
% the training options for the experiment. Read the hyperparameters specified 
% in Experiment Manager by accessing the fields of the input |params|. For example, 
% access the initial learning rate |myInitialLearnRate| defined in the hyperparameter 
% table with the argument value |params.myInitialLearnRate|.
options = trainingOptions('sgdm', ...
    'MaxEpochs',5, ... 
    'ValidationData',{XValidation,TValidation}, ...
    'ValidationFrequency',30, ...
    'InitialLearnRate',params.myInitialLearnRate);
%% 
% End of experiment setup function.
end
%% 
% Copyright 2020-2021 The MathWorks, Inc.