%% Calculate Size of Tall Array
% Convert a datastore into a tall table, calculate its size using a deferred calculation, and then perform the calculation and return the result in memory.
% First, create a datastore for the airlinesmall.csv data set. Treat 'NA' values as missing data so that they are replaced with NaN values. Set the text format of a few columns so that they are read as a cell array of character vectors. Convert the datastore into a tall table.

ds = tabularTextDatastore('airlinesmall.csv', 'TreatAsMissing', 'NA');
ds.SelectedFormats{strcmp(ds.SelectedVariableNames, 'TailNum')} = '%s';
ds.SelectedFormats{strcmp(ds.SelectedVariableNames, 'CancellationCode')} = '%s';

%

T = tall(ds)

% The display of the tall table indicates that MATLAB® does not yet know how many rows of data are in the table.
%
% Calculate the size of the tall table. Since calculating the size of a tall array requires a full pass through the data, MATLAB does not immediately calculate the value. Instead, like most operations with tall arrays, the result is an unevaluated tall array whose values and size are currently unknown.

s = size(T)

%
% Use the gather function to perform the deferred calculation and return the result in memory. The result returned by size is a trivially small 1-by-2 vector, which fits in memory.

sz = gather(s)

% If you use gather on an unreduced tall array, then the result might not fit in memory. If you are unsure whether the result returned by gather can fit in memory, use gather(head(X)) or gather(tail(X)) to bring only a small portion of the calculation result into memory.
% Copyright 2019 The MathWorks, Inc.