Compare data in two simulation runs
compares the data in the runs that correspond to diffResult
= Simulink.sdi.compareRuns(runID1
,runID2
)runID1
and
runID2
and returns the result in the Simulink.sdi.DiffRunResult
object
diffResult
. The comparison uses the Simulation Data Inspector
comparison algorithm. For more information about the algorithm, see How the Simulation Data Inspector Compares Data.
compares the simulation runs that correspond to diffResult
= Simulink.sdi.compareRuns(runID1
,runID2
,Name,Value
)runID1
and
runID2
using the options specified by one or more
Name,Value
pair arguments. For more information about how the
options can affect the comparison, see How the Simulation Data Inspector Compares Data.
This example shows how to compare runs of simulation data and then analyze and save the results using the Simulation Data Inspector programmatic interface.
Create Simulation Data
First, create simulation data by simulating a model that logs data. This example uses the ex_sldemo_absbrake
model and analyzes the effect of changing the Desired relative slip
value.
Load the model. Use the set_param
function to specify an initial value for the relative slip and simulate the model.
load_system('ex_sldemo_absbrake') set_param('ex_sldemo_absbrake/Desired relative slip','Value','0.24') out_1 = sim('ex_sldemo_absbrake');
Use the set_param
function to specify a different value for the relative slip and simulate the model again.
set_param('ex_sldemo_absbrake/Desired relative slip','Value','0.25') out_2 = sim('ex_sldemo_absbrake');
Compare Runs Using Global Tolerance Values
First, use the Simulink.sdi.getAllRunIDs
function to get the run IDs that correspond to the last two simulation runs.
runIDs = Simulink.sdi.getAllRunIDs; runID1 = runIDs(end - 1); runID2 = runIDs(end);
Now, use the Simulink.sdi.compareRuns
function to compare the runs. Specify a global relative tolerance value of 0.2
and a global time tolerance value of 0.5
.
runResult = Simulink.sdi.compareRuns(runID1,runID2,'reltol',0.2,'timetol',0.5);
Check the Summary
property of the returned Simulink.sdi.DiffRunResult
object.
runResult.Summary
ans = struct with fields:
OutOfTolerance: 2
WithinTolerance: 2
Unaligned: 0
UnitsMismatch: 0
Empty: 0
Canceled: 0
EmptySynced: 0
DataTypeMismatch: 0
TimeMismatch: 0
StartStopMismatch: 0
Unsupported: 0
Two signal comparisons within the run were within tolerance, and two were out of tolerance.
Plot Comparison Results
You can use plots to analyze the comparison results. Access the signal result for the Ww
signal from the DiffRunResult
object that contains the comparison results using the getResultByIndex
function. Check the Status
property of the Simulink.sdi.DiffSignalResult
object.
signalResult_Ww = getResultByIndex(runResult,1)
signalResult_Ww = DiffSignalResult with properties: Name: 'yout.Ww' Status: OutOfTolerance AlignBy: 'Path' SignalID1: 127816 SignalID2: 127862 MaxDifference: 12.4878 Sync1: [1x1 timeseries] Sync2: [1x1 timeseries] Diff: [1x1 timeseries]
signalResult_Ww.Status
ans = OutOfTolerance
The Ww
signal comparison results are out of tolerance. Plot the difference signal to analyze the result.
figure(1) plot(signalResult_Ww.Diff)
Save Comparison Results
You can save the comparison results to an MLDATX file to analyze later or to share with a colleague. Use the saveResult
function to save the run data and comparison results.
saveResult(runResult,'desiredSlipResults')
The MLDATX file desiredSlipResults
is created in the working directory. Use the Simulink.sdi.load
function or the open
function to view the results in the MLDATX file.
Using the Simulation Data Inspector programmatic interface, you can specify signal tolerance values to use in comparisons. This example uses the slexAircraftExample
model and the Simulation Data Inspector to evaluate the effect of changing the time constant for the low-pass filter following the control input.
Configure the Model
Load the model and mark signals of interest for logging. This example logs data for the q
and alpha
signals.
load_system('slexAircraftExample') Simulink.sdi.markSignalForStreaming('slexAircraftExample/Aircraft Dynamics Model',3,'on') Simulink.sdi.markSignalForStreaming('slexAircraftExample/Aircraft Dynamics Model',4,'on')
Run Simulations
Run simulations with different low-pass filter time constants to generate results to compare. The slexAircraftExample
model stores variables associated with the model in the model workspace. To modify the time constant value, access the model workspace and use the assignin
function.
out1 = sim('slexAircraftExample'); modelWorkspace = get_param('slexAircraftExample','modelworkspace'); assignin(modelWorkspace,'Ts',1) out2 = sim('slexAircraftExample');
Access and Compare Simulation Results
Access the simulation results using the Simulation Data Inspector programmatic interface. Each simulation creates a run in the Simulation Data Inspector with a unique run ID. You use the run IDs to compare the simulation results.
runIDs = Simulink.sdi.getAllRunIDs; runIDTs1 = runIDs(end-1); runIDTs2 = runIDs(end);
Use the Simulink.sdi.compareRuns
function to compare the data from the simulations. Then inspect the Status
property of the signal result to see whether the signals fell within the default tolerance of 0.
diffRun1 = Simulink.sdi.compareRuns(runIDTs1,runIDTs2); sig1Result1 = getResultByIndex(diffRun1,1); sig2Result1 = getResultByIndex(diffRun1,2); sig1Result1.Status
ans = OutOfTolerance
sig2Result1.Status
ans = OutOfTolerance
Compare Runs with Signal Tolerances
By default, signals use 0
for all tolerance values, so the comparison returns out-of-tolerance results when the signals are not identical. To further analyze the effect of the time constant change, specify tolerance values for the signals. You can specify tolerances for a programmatic comparison using the properties of the Simulink.sdi.Signal
objects in the runs you compare. The comparison uses the tolerances specified for the baseline Signal
object. This example specifies a combination of time and absolute tolerances.
To specify tolerances, first access the Simulink.sdi.Signal
objects that correspond to each signal in the runs you want to compare.
run1 = Simulink.sdi.getRun(runIDTs1); sigID1 = getSignalIDByIndex(run1,1); sigID2 = getSignalIDByIndex(run1,2); sig1 = Simulink.sdi.getSignal(sigID1); sig2 = Simulink.sdi.getSignal(sigID2);
Check the Name
property to identify each Signal
object.
sig1.Name
ans = 'q, rad/sec'
sig2.Name
ans = 'alpha, rad'
Specify an absolute tolerance of 0.1
and a time tolerance of 0.6
for the q
signal using the AbsTol
and TimeTol
properties of the q
signal object in the baseline run.
sig1.AbsTol = 0.1; sig1.TimeTol = 0.6;
Specify an absolute tolerance of 0.2
and a time tolerance of 0.8
for the alpha
signal using the AbsTol
and TimeTol
properties of the alpha
signal object in the baseline run.
sig2.AbsTol = 0.2; sig2.TimeTol = 0.8;
Compare the runs again and access the results.
diffRun2 = Simulink.sdi.compareRuns(runIDTs1,runIDTs2); sig1Result2 = getResultByIndex(diffRun2,1); sig2Result2 = getResultByIndex(diffRun2,2);
Check the Status
property of each signal to determine whether the comparison results fell within the specified tolerances.
sig1Result2.Status
ans = WithinTolerance
sig2Result2.Status
ans = WithinTolerance
You can use the Simulink.sdi.compareRuns
function to compare signal data and metadata, including data type and start and stop times. A single comparison may check for mismatches in one or more pieces of metadata. When you check for mismatches in signal metadata, the Summary
property of the Simulink.sdi.DiffRunResult
object may differ from a basic comparison because the Status
property for a Simulink.sdi.DiffSignalResult
object can indicate the metadata mismatch. You can configure comparisons using the Simulink.sdi.compareRuns
function for imported data and for data logged from a simulation.
This example configures a comparison of runs created from workspace data three ways to show how the Summary
of the DiffSignalResult
object can provide specific information about signal mismatches.
Create Workspace Data
The Simulink.sdi.compareRuns
function compares time series data. Create data for a sine wave to use as the baseline signal, using the timeseries
format. Give the timeseries
the name Wave Data
.
time = 0:0.1:20;
sig1vals = sin(2*pi/5*time);
sig1_ts = timeseries(sig1vals,time);
sig1_ts.Name = 'Wave Data';
Create a second sine wave to compare against the baseline signal. Use a slightly different time vector and attenuate the signal so the two signals are not identical. Cast the signal data to the single
data type. Also name this timeseries
object Wave Data
. The Simulation Data Inspector comparison algorithm will align these signals for comparison using the name.
time2 = 0:0.1:22;
sig2vals = single(0.98*sin(2*pi/5*time2));
sig2_ts = timeseries(sig2vals,time2);
sig2_ts.Name = 'Wave Data';
Create and Compare Runs in the Simulation Data Inspector
The Simulink.sdi.compareRuns
function compares data contained in Simulink.sdi.Run
objects. Use the Simulink.sdi.createRun
function to create runs in the Simulation Data Inspector for the data. The Simulink.sdi.createRun
function returns the run ID for each created run.
runID1 = Simulink.sdi.createRun('Baseline Run','vars',sig1_ts); runID2 = Simulink.sdi.createRun('Compare to Run','vars',sig2_ts);
You can use the Simulink.sdi.compareRuns
function to compare the runs. The comparison algorithm converts the signal data to the double
data type and synchronizes the signal data before computing the difference signal.
basic_DRR = Simulink.sdi.compareRuns(runID1,runID2);
Check the Summary
property of the returned Simulink.sdi.DiffRunResult
object to see the result of the comparison.
basic_DRR.Summary
ans = struct with fields:
OutOfTolerance: 1
WithinTolerance: 0
Unaligned: 0
UnitsMismatch: 0
Empty: 0
Canceled: 0
EmptySynced: 0
DataTypeMismatch: 0
TimeMismatch: 0
StartStopMismatch: 0
Unsupported: 0
The difference between the signals is out of tolerance.
Compare Runs and Check for Data Type Match
Depending on your system requirements, you may want the data types for signals you compare to match. You can use the Simulink.sdi.compareRuns
function to configure the comparison algorithm to check for and report data type mismatches.
dataType_DRR = Simulink.sdi.compareRuns(runID1,runID2,'DataType','MustMatch'); dataType_DRR.Summary
ans = struct with fields:
OutOfTolerance: 0
WithinTolerance: 0
Unaligned: 0
UnitsMismatch: 0
Empty: 0
Canceled: 0
EmptySynced: 0
DataTypeMismatch: 1
TimeMismatch: 0
StartStopMismatch: 0
Unsupported: 0
The result of the signal comparison is now DataTypeMismatch
because the data for the baseline signal is double
data type, while the data for the signal compared to the baseline is single
data type.
Compare Runs and Check for Start and Stop Time Match
You can use the Simulink.sdi.compareRuns
function to configure the comparison algorithm to check whether the aligned signals have the same start and stop times.
startStop_DRR = Simulink.sdi.compareRuns(runID1,runID2,'StartStop','MustMatch'); startStop_DRR.Summary
ans = struct with fields:
OutOfTolerance: 0
WithinTolerance: 0
Unaligned: 0
UnitsMismatch: 0
Empty: 0
Canceled: 0
EmptySynced: 0
DataTypeMismatch: 0
TimeMismatch: 0
StartStopMismatch: 1
Unsupported: 0
The signal comparison result is now StartStopMismatch
because the signals created in the workspace have different stop times.
This example shows how to compare runs using your desired criteria for aligning signals between runs.
Generate Simulation Data to Compare
Start by loading the slexAircraftExample
system and marking signals of interest for logging.
load_system('slexAircraftExample') Simulink.sdi.markSignalForStreaming('slexAircraftExample/Pilot',1,'on') Simulink.sdi.markSignalForStreaming('slexAircraftExample/Aircraft Dynamics Model',... 4,'on')
This example runs two simulations of the slexAircraftExample
model with different values for the variable Ts
. Use get_param
to access the model workspace for the slexAircraftExample
model. Then, assign a value for the variable and run the first simulation.
modelWorkspace = get_param('slexAircraftExample','modelworkspace'); modelWorkspace.assignin('Ts',0.1); out_1 = sim('slexAircraftExample');
Change the value of Ts
and run the second simulation. Then,
modelWorkspace.assignin('Ts',0.2); out_2 = sim('slexAircraftExample');
Use the Simulink.sdi.getAllRunIDs
function to access the run IDs for the two simulation runs created when you simulated the slexAircraftExample
model.
runIDs = Simulink.sdi.getAllRunIDs; runID1 = runIDs(end-1); runID2 = runIDs(end);
Define Alignment Criteria for the Comparison
Before running the comparison, define how you want the Simulation Data Inspector to align the signals between the runs. This example aligns signals by their name, then by their block path, and then by their Simulink identifier.
alignMethods = [Simulink.sdi.AlignType.SignalName Simulink.sdi.AlignType.BlockPath Simulink.sdi.AlignType.SID];
Compare the Runs with the Specified Alignment Criteria
Compare the simulation data in your two runs, using the alignment criteria you specified. The comparison uses a small time tolerance to account for the effect of differences in the step size used by the solver on the transition of the square wave input.
diffResults = Simulink.sdi.compareRuns(runID1,runID2,'align',alignMethods,... 'timetol',0.005);
Check the Comparison Results for the Aligned Signals
You can use the getResultByIndex
method to access the comparison results for the aligned signals in the runs you compared. You can use the Count
property of the Simulink.sdi.DiffRunResult
object to set up a for
loop to check the Status
property for each Simulink.sdi.DiffSignalResult
object.
numComparisons = diffResults.count; for k = 1:numComparisons resultAtIdx = getResultByIndex(diffResults,k); sigID1 = resultAtIdx.signalID1; sigID2 = resultAtIdx.signalID2; sig1 = Simulink.sdi.getSignal(sigID1); sig2 = Simulink.sdi.getSignal(sigID2); displayStr = 'Signals %s and %s: %s \n'; fprintf(displayStr,sig1.Name,sig2.Name,resultAtIdx.Status); end
Signals alpha, rad and alpha, rad: OutOfTolerance Signals Stick and Stick: WithinTolerance
runID1
— Baseline run identifierNumeric identifier for the baseline run in the comparison, specified as a
run ID that corresponds to a run in the Simulation Data Inspector. The
Simulation Data Inspector assigns run IDs when runs are created. You can get
the run ID for a run by using the ID
property of the
Simulink.sdi.Run
object, the
Simulink.sdi.getAllRunIDs
function, or the Simulink.sdi.getRunIDByIndex
function.
runID2
— Identifier for run to compareNumeric identifier for the run to compare, specified as a run ID that
corresponds to a run in the Simulation Data Inspector. The Simulation Data
Inspector assigns run IDs when runs are created. You can get the run ID for
a run by using the ID
property of the Simulink.sdi.Run
object, the
Simulink.sdi.getAllRunIDs
function, or the Simulink.sdi.getRunIDByIndex
function.
Specify optional
comma-separated pairs of Name,Value
arguments. Name
is
the argument name and Value
is the corresponding value.
Name
must appear inside quotes. You can specify several name and value
pair arguments in any order as
Name1,Value1,...,NameN,ValueN
.
'abstol',x,'align',alignOpts
'Align'
— Signal alignment optionsSignal alignment options, specified as the comma-separated pair
consisting of 'Align'
and a string array or array of
character vectors.
Array specifying alignment options to use for pairing signals from the runs being compared. The Simulation Data Inspector aligns signals first by the first element in the array, then by the second element in the array, and so on. For more information, see Signal Alignment.
Value | Aligns By |
---|---|
Simulink.sdi.AlignType.BlockPath | Path to the source block for the signal |
Simulink.sdi.AlignType.SID | Simulink® identifier Simulink Identifiers |
Simulink.sdi.AlignType.SignalName | Signal name |
Simulink.sdi.AlignType.DataSource | Path of the variable in the MATLAB® workspace |
Example: [Simulink.sdi.AlignType.SignalName,Simulink.sdi.AlignType.SID]
specifies signal alignment by name and then by SID.
'AbsTol'
— Absolute tolerance for comparison0
(default) | scalarPositive-valued global absolute tolerance used for all signals in the
run comparison, specified as the comma-separated pair consisting of
'AbsTol'
and a scalar. For more information about
how tolerances are used in comparisons, see Tolerance Specification.
Example: 0.5
Data Types: double
'RelTol'
— Relative tolerance for comparison0
(default) | scalarPositive-valued global relative tolerance used for all signals in the
run comparison, specified as the comma-separated pair consisting of
'RelTol'
and a scalar. The relative tolerance is
expressed as a fractional multiplier. For example,
0.1
specifies a 10 percent tolerance. For more
information about how the relative tolerance is applied in the
Simulation Data Inspector, see Tolerance Specification.
Example: 0.1
Data Types: double
'TimeTol'
— Time tolerance for comparison0
(default) | scalarPositive-valued global time tolerance used for all signals in the run
comparison, specified as the comma-separated pair consisting of
'TimeTol'
and a scalar. Specify the time
tolerance in units of seconds. For more information about tolerances in
the Simulation Data Inspector, see Tolerance Specification.
Example: 0.2
Data Types: double
'DataType'
— Comparison sensitivity to signal data types'MustMatch'
Specify the name-value pair 'DataType','MustMatch'
when you want the comparison to be sensitive to data type mismatches in
compared signals. When you specify this name-value pair, the algorithm
compares the data types for aligned signals before synchronizing and
comparing the signal data.
The Simulink.sdi.compareRuns
function does not
compare the data types of aligned signals unless you specify this
name-value pair. The comparison algorithm can compare signals with
different data types.
When signal data types do not match, the Status
property of the Simulink.sdi.DiffSignalResult
object for the result is set
to DataTypeMismatch
.
When you specify that data types must match and configure the comparison to stop on the first mismatch, a data type mismatch stops the comparison. A stopped comparison may not compute results for all signals.
'Time'
— Comparison sensitivity to signal time vectors'MustMatch'
Specify the name-value pair 'Time','MustMatch'
when
you want the comparison to be sensitive to mismatches in the time
vectors of compared signals. When you specify this name-value pair, the
algorithm compares the time vectors of aligned signals before
synchronizing and comparing the signal data.
Comparisons are not sensitive to differences in signal time vectors unless you specify this name-value pair. For comparisons that are not sensitive to differences in the time vectors, the comparison algorithm synchronizes the signals prior to the comparison. For more information about how synchronization works, see How the Simulation Data Inspector Compares Data.
When the time vectors for signals do not match, the
Status
property of the Simulink.sdi.DiffSignalResult
object for the result is set
to TimeMismatch
.
When you specify that time vectors must match and configure the comparison to stop on the first mismatch, a time vector mismatch stops the comparison. A stopped comparison may not compute results for all signals.
'StartStop'
— Comparison sensitivity to signal start and stop times'MustMatch'
Specify the name-value pair 'StartStop','MustMatch'
when you want the comparison to be sensitive to mismatches in signal
start and stop times. When you specify this name-value pair, the
algorithm compares the start and stop times for aligned signals before
synchronizing and comparing the signal data.
When the start times and stop times do not match, the
Status
property of the Simulink.sdi.DiffSignalResult
object for the result is set
to StartStopMismatch
.
When you specify that start and stop times must match and configure the comparison to stop on the first mismatch, a start or stop time mismatch stops the comparison. A stopped comparison may not compute results for all signals.
'StopOnFirstMismatch'
— Whether comparison stops on first detected mismatch'Metadata'
| 'Any'
Whether the comparison stops without comparing remaining signals on
the first detected mismatch, specified as the comma-separated pair
consisting of 'StopOnFirstMismatch'
and
'Metadata'
or 'Any'
. A stopped
comparison may not compute results for all signals, and can return a
mismatched result more quickly.
Metadata
— A mismatch in metadata for
aligned signals causes the comparison to stop. Metadata
comparisons happen before comparing signal data.
The Simulation Data Inspector always aligns signals and compares signal units. When you configure the comparison to stop on the first mismatch, an unaligned signal or mismatched units always causes the comparison to stop. You can specify additional name-value pairs to configure the comparison to check and stop on the first mismatch for additional metadata, such as signal data type, start and stop times, and time vectors.
Any
— A mismatch in metadata or signal
data for aligned signals causes the comparison to
stop.
diffResult
— Comparison resultsSimulink.sdi.DiffRunResult
Comparison results, returned as a Simulink.sdi.DiffRunResult
object.
getResultByIndex
| Simulink.sdi.compareSignals
| Simulink.sdi.DiffRunResult
| Simulink.sdi.DiffSignalResult
| Simulink.sdi.getRunCount
| Simulink.sdi.getRunIDByIndex
You have a modified version of this example. Do you want to open this example with your edits?