%% Define Custom Mini-Batch Datastore For Super-Resolution Networks
% This example shows how to define a custom mini-batch datastore that can be used to feed training data to a deep learning network. The custom mini-batch datastore, called vdsrImagePatchDatastore, is a convenient way to generate augmented image patches for training a VDSR network. The datastore is attached to this example as a supporting file, vdsrImagePatchDatastore.m.
% The input to a vdsrImagePatchDatastore is an image datastore that contains high-resolution images. The datastore goes through these steps to generate each mini-batch of training data:
% Convert each image to the YCbCr color space and extracts the luminance (Y) channel. The result is a high-resolution reference image.
% Create a low-resolution image by downscaling the reference image by a scale factor. Then, upscale the low-resolution image using bicubic interpolation to the original image size.
% Crop the reference image to a fixed patch size from a random location in the image. Apply an identical crop to the upscaled low-resolution image.
% Calculate the difference between the reference image patch and the upscaled low-resolution image patch. The result is the residual image patch.
% Define augmentation options using an imageDataAugmenter. For this example, supported transformations include random horizontal and vertical reflection and random rotation by 90 degrees.
% Apply a random transformation to the upscaled low-resolution image patch and an identical transformation to the residual image patch.
% Create a table that contains the training data. The augmented upscaled low-resolution image patches act as the network input. Store these as the first variable in the table. The augmented residual image patches act as the desired network responses. Store these as the second variable in the table.
%
% To explore how the datastore generates mini-batches, create a vdsrImagePatchDatastore.
% First, create an imageDatastore that contains high-resolution images. This image datastore, testImages, contains 21 undistorted images shipped in Image Processing Toolbox™.

exts = {'.jpg','.png'};
fileNames = {'sherlock.jpg','car2.jpg','fabric.png','greens.jpg','hands1.jpg','kobi.png',...
    'lighthouse.png','micromarket.jpg','office_4.jpg','onion.png','pears.png','yellowlily.jpg',...
    'indiancorn.jpg','flamingos.jpg','sevilla.jpg','llama.jpg','parkavenue.jpg',...
    'peacock.jpg','car1.jpg','strawberries.jpg','wagon.jpg'};
filePath = [fullfile(matlabroot,'toolbox','images','imdata') filesep];
filePathNames = strcat(filePath,fileNames);
testImages = imageDatastore(filePathNames,'FileExtensions',exts);

% Create a vdsrImagePatchDatastore that processes the images in testImages. Specify that each mini-batch consists of 64 patches of size 41-by-41 pixels. Extract one mini-batch from each image during training. Set 'BatchesPerImage' to the number of random patches to be extracted per image per mini-batch. To train a multiscale factor network, set 'ScaleFactor' to [2 3 4].

miniBatchSize = 64;
scaleFactors = [2 3 4];
source = vdsrImagePatchDatastore(testImages, ...
    'MiniBatchSize',miniBatchSize, ...
    'PatchSize',41, ...
    'BatchesPerImage',1, ...
    'ScaleFactor',scaleFactors);

%
%  Perform a read operation on the datastore to explore the data.

inputBatch = read(source);
summary(inputBatch)

% Display the mini-batch of upscaled low-resolution patches generated by the vdsrImagePatchDatastore.

montage(inputBatch.upsampledPatches)

% Display the mini-batch of residual patches generated by the vdsrImagePatchDatastore. The magnitude of the residuals is small, so decrease the display range to amplify the displayed residuals.

montage(inputBatch.residualPatches,'DisplayRange',[0 0.05])

% Copyright 2018 The MathWorks, Inc.