%% Create Simple Deep Learning Network for Classification Using Deep Network Designer
% This example shows how to create and train a simple convolutional neural network for deep learning classification using the Deep Network Designer app. Convolutional neural networks are essential tools for deep learning, and are especially suited for image recognition.
% The example demonstrates how to:
% Import image data.
% Define the network architecture.
% Specify training options.
% Train the network.
%% Load Image Data
% Load the digit sample data as an image datastore. imageDatastore automatically labels the images based on folder names and stores the data as an ImageDatastore object. An image datastore enables you to store large image data, including data that does not fit in memory, and efficiently read batches of images during training of a convolutional neural network.

digitDatasetPath = fullfile(matlabroot,'toolbox','nnet','nndemos', ...
    'nndatasets','DigitDataset');
imds = imageDatastore(digitDatasetPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');

% Open Deep Network Designer. Create a network, import and visualize data, and train using Deep Network Designer.

deepNetworkDesigner

% Pause over Blank Network and click New.
% To import the image datastore, select the Data tab and click Import Data. Select imds as the data souce. Set aside 30% of the training data to be used as validation data. Select Randomize to randomly allocate observations to the training and validations sets.
%
% To import the data, click Import.
% Deep Network Designer displays a histogram of the class labels. You can see that the training data has 7000 observations across 10 class with each class containing 700 observations. You can also see a randomly selection of images from each class.
%
%
%% Define Network Architecture
% In the Designer pane, define the convolutional neural network architecture. Drag layers from the layer library and connect them. To edit the properties of a layer, click the layer and edit the values in the Property Panel.
%
%
% 1.  Image Input Layer with InputSize 28,28,1
% An imageInputLayer is where you specify the image size, which, in this case, is 28-by-28-by-1. These numbers correspond to the height, width, and the channel size. The digit data consists of grayscale images, so the channel size (color channel) is 1. For a color image, the channel size is 3, corresponding to the RGB values.
% 2. 2D Convolutional Layer with NumFilters 8
% In the convolutional layer the FilterSize is the height and width of the filters the training function uses while scanning along the images. In this example, 3,3 indicates that the filter size is 3-by-3. The property NumFilters is the number of filters, in this example set NumFilters to 8. This parameter determines the number of feature maps. Use the 'Padding' property to add padding to the input feature map. For a convolutional layer with a default stride of 1, 'same' padding ensures that the spatial output size is the same as the input size.
% 3. Batch Normalization Layer
% Batch normalization layers normalize the activations and gradients propagating through a network, making network training an easier optimization problem. Use batch normalization layers between convolutional layers and nonlinearities, such as ReLU layers, to speed up network training and reduce the sensitivity to network initialization.
% 4. ReLU Layer
% The batch normalization layer is followed by a nonlinear activation function. The most common activation function is the rectified linear unit (ReLU).
% 5. Max Pooling Layer with PoolSize 2 and Stride 2
% Convolutional layers (with activation functions) are sometimes followed by a down-sampling operation that reduces the spatial size of the feature map and removes redundant spatial information. Down-sampling makes it possible to increase the number of filters in deeper convolutional layers without increasing the required amount of computation per layer. One way of down-sampling is using a max pooling layer. The max pooling layer returns the maximum values of rectangular regions of inputs, specified by the PoolSize. In this example, set PoolSize to 2. The Stride property specifies the step size that the training function takes as it scans along the input, in this example set this to 2.
% 6. Fully Connected Layer with OutputSize 10
% The convolutional and down-sampling layers are followed by one or more fully connected layers. As its name suggests, a fully connected layer is a layer in which the neurons connect to all the neurons in the preceding layer. This layer combines all the features learned by the previous layers across the image to identify the larger patterns. The last fully connected layer combines the features to classify the images. Therefore, the OutputSize property in the last fully connected layer is equal to the number of classes in the target data. In this example, the output size is 10, corresponding to the 10 classes.
% 7. Softmax Layer
% The softmax activation function normalizes the output of the fully connected layer. The output of the softmax layer consists of positive numbers that sum to one, which can then be used as classification probabilities by the classification layer.
% 8. Classification Layer
% The final layer is the classification layer. This layer uses the probabilities returned by the softmax activation function for each input to assign the input to one of the mutually exclusive classes and compute the loss.
%
%% Specify Training Options
% After defining the network structure, to set training options select the Training tab and click Training Options.
% Train the network using stochastic gradient descent with momentum (SGDM) with an initial learning rate of 0.01. Set the maximum number of epochs to 5. An epoch is a full training cycle on the entire training data set. The software trains the network on the training data and calculates the accuracy on the validation data at regular intervals during training. The validation data is not used to update the network weights.
%
%
%% Train Network Using Training Data
% To train the network using the network architecture defined, the training data, and the training options, click Train. By default, Deep Network Designer uses a GPU if one is available. This requires Parallel Computing Toolbox™ and a supported GPU device. For information on supported devices, see GPU Support by Release. Otherwise, it uses a CPU. You can also specify the execution environment by setting 'ExecutionEnvironment' in the Training Options.
% The training progress plot shows the mini-batch loss and accuracy and the validation loss and accuracy. For more information on the training progress plot, see Monitor Deep Learning Training Progress. The loss is the cross-entropy loss. The accuracy is the percentage of images that the network classifies correctly.
%
% Accuracy is the fraction of labels that the network predicts correctly. In this case, more than 95% of the predicted labels match the true labels of the validation set.
% To export the trained network, in the Training tab, click Export.
% Copyright 2020 The MathWorks, Inc.