%% Data Analytics Application with Many MDF-Files
% This example shows you how to investigate vehicle battery power during discharge mode across various drive cycles. The data for this analysis are contained in a set of vehicle log files in MDF format. For this example, we need to build up a mechanism that can "detect" when the vehicle battery is in a given mode. What we are really doing is building a detector to determine when a signal of interest (battery power in this case) meets specific criteria. When the criteria is met, we will call that an "event".  Each event will be subsequently "qualified" by imposing time bounds. That is to say an event is "qualified" if it persists for at least 5 seconds (such a qualification step can help limit noise and remove transients). The thresholds shown in this example are illustrative only.
%
%% Set Data Source Location
% Define the location of the file set to analyze.

dataDir = '*.dat';

%
%% Obtain File Set Information
% Get the names of all the MDF-files to analyze into a single cell array.

fileList = dir(dataDir);
fileName = {fileList(:).name}';
fileDir  = {fileList(:).folder}';
fullFilePath = fullfile(fileDir, fileName)

%
%% Pre-allocate the Output Data Cell Array
% Use a cell array to capture a collection of mini-tables which represent the event data of interest for each individual MDF-file.

numFiles = size(fullFilePath, 1);
eventSet = cell(numFiles, 1)

%
%% Define Event Detection and Channel Information Criteria

chName = 'Power';         % Name of the signal of interest in the MDF-files
thdValue = [5, 55];       % Threshold in KW
thdDuration = seconds(5); % Threshold for event qualification

%
%% Loop Through Each MDF-File and Apply the Event Detector Function
% eventSet is a cell array which contains a summary table for each file that was analyzed. You can think of this cell array of tables as a set of mini-tables, all with the same format but the contents of each mini-table correspond to the individual MDF-files.
% In this example, the event detector not only reports the event start and end times but also some descriptive statistics about the event itself. This kind of aggregation and reporting can be useful for discovery and troubleshooting activities. To understand the MDF-file interfacing and data handling in more detail, open and explore the processMDF function from this example.
% Note that the data processing is written such that each MDF-file is parsed atomically and returns into its own index of the resulting cell array. This allows the processing function to leverage parallel computing capability with parfor. parfor and standard for are interchangeable in terms of outputs, but result in varying processing time needed to complete the analysis. To experiment with parallel computing, simply change the for call below to parfor and run this example.

for i = 1:numFiles
    eventSet{i} = processMDF(fullFilePath{i}, chName, thdValue, thdDuration);
end
eventSet{1}

%
%% Concatenate Results
% Combine the contents of the cell array eventSet into a single table. We can now use the table eventSummary for subsequent analysis. The head function is used to display the first 5 rows of the table eventSummary.

eventSummary = vertcat(eventSet{:});
disp(head(eventSummary, 5))

%
%% Visualize Summary Results to Determine Next Steps
% Look at an overview of the event durations.

histogram(eventSummary.EventDuration)
grid on
title 'Distribution of Event Duration'
xlabel 'Event Duration (minutes)'
ylabel 'Frequency'

% Now look at Mean Power vs. Event Duration.

scatter(eventSummary.MeanPower_KW, minutes(eventSummary.EventDuration))
grid on
xlabel 'MeanPower(KW)'
ylabel 'Event Duration (minutes)'
title 'Mean Power vs. Event Duration'

%
%% Deep Dive an Event of Interest
% Inspect the event that lasted for more than 4 minutes. First, create a mask to find the case of interest. msk is a logical index that shows which rows of the table eventSummary meet the specified criteria.

msk = eventSummary.EventDuration > minutes(4);

% Pull out the rows of the table eventSummary that meet the criteria specified and display the results.

eventOfInterest = eventSummary(msk, :);
disp(eventOfInterest)

%
%% Visualize This Event in the Context of the Entire Drive Cycle
% We need the full file path and file name to read the data from the MDF-file. The table eventOfInterest has the filename because we kept track of that. It does not have the full file path to that file. To get this information we will apply a bit of set theory to our original list of filenames and paths. First, find the full file path of the file of interest.

fileMsk = find(ismember(fileName, eventOfInterest.FileName))

% Create an MDF object to read data from the MDF-file.

mdfObj = mdf(fullFilePath{fileMsk})

% Identify the channel with channelList and read all the data from this file.

chInfo = channelList(mdfObj, chName)
data = read(mdfObj, chInfo)

% Note that reading with the output of channelList returns a cell array of results.

data{1}(1:10,:)

%
%% Visualize Using a Custom Plotting Function
% Custom plotting functions are useful for encapsulation and reuse. Visualize the event in the context of the entire drive cycle. To understand how the visualization was created, open and explore the eventPlotter function from this example.

eventPlotter(data{1}, eventOfInterest)

%
%% Close the File
% Close access to the MDF-file by clearing its variable from the workspace.

clear mdfObj

%
% Copyright 2018-2020 The MathWorks, Inc.