Test Iterations

You can run the same test case with different data or configuration sets by using test case iterations. Iterations can use different:

  • Parameters.

  • External inputs.

  • Configuration sets.

  • Signal Editor scenarios.

  • Signal Builder groups.

  • Baseline data.

Set up iterations in the Iterations section of a test case. You can use table iterations or scripted iterations. If the test collects coverage using Simulink® Coverage™, the same coverage settings apply to all iterations in the test case.

Whether you use table or scripted iterations, you can see the iterations in the test case by clicking the Show Iterations button.

Create Table Iterations

Table Iterations provide a quick way to add iterations based items in your model or test case. To create iterations with the table, first make the appropriate columns visible:

  1. Expand the Iterations > Table Iterations section.

  2. In the table, add or remove columns by clicking the button and selecting items in the list. For example, to display parameter and configuration sets, select the Parameter Set and Configuration Set items.

Add Iterations Manually

  1. To manually add iterations, click Add. The table displays a new iteration row.

  2. Assign an iteration name and select items for the iteration. For example, this test case has four iterations. Each iteration uses a different combination of external input and baseline data.

Generate Table Iterations

You can also automatically generate iterations from data in your test case and model:

  1. Click the Auto Generate button.

  2. Select items to generate iterations.

    If you select multiple items, iterations are created in sequential pairings. For example:

    • The model sldemo_autotrans has a Signal Builder block with four signal groups, labeled S1, S2, S3, and S4.

    • The test case has three parameter sets, labeled P1, P2, and P3.

    • Automatically generating iterations from Signal Builder groups and parameter sets results in three iterations. The iterations are limited by the three parameter sets. Each iteration contains one Signal Builder group and one parameter set. The Signal Builder group and parameter set are matched in the order that they are listed in the Signal Builder block and parameter set section.

    Signal Builder, parameter, and iteration blocks.

  3. Specify an optional naming rule for the iterations. In the Iteration naming rule box, enter the rule using:

    • The name of each setting you want to use in the name, with spaces removed

    • An underscore or space to separate each setting

    For example, if you want to include the name of the parameter set, configuration set, and baseline file name, enter ParameterSet_ConfigurationSet_Baseline.

Section OptionPurpose

Signal Builder Group

Applies to the Inputs section of a simulation, baseline, or equivalence test case, for the specified Signal Builder Group. Each Signal Builder group is used to generate an iteration.

Signal Editor scenario

Applies to the Inputs section of a simulation, baseline, or equivalence test case, for the specified Signal Editor Scenario. Each Signal Editor scenario is used to generate an iteration.

Parameter Set

Applies to the Parameter Overrides section of a simulation, baseline, or equivalence test case. Each parameter override set is used to generate an iteration.

External Input

Applies to the Inputs section of a simulation, baseline, or equivalence test case, for the specified External Inputs sets. Each external input set is used to generate an iteration.

Configuration Set

Applies to the Configuration Setting Overrides section of a simulation, baseline, or equivalence test case. Each iteration uses the configuration setting specified.

Baseline

Applies only to baseline test case types, specifically to the Baseline Criteria section of a baseline test case. Each baseline criteria set is used to generate an iteration.

Simulation 1 or 2

Applies only to equivalence test case types. At the top of the Auto Generate Reports dialog box, there is a menu for Simulation 1 or Simulation 2. These sections correspond to the two simulation sections within the equivalence test case.

Create Scripted Iterations

You can run a custom set of iterations using a script in the Scripted Iterations section. For example, you can define parameter sets or customize iteration order by using a custom iteration. Scripted iterations are generated at run time when a test executes.

Iteration Script Components

An iteration script must contain certain components. The most basic iteration script contains three elements:

  1. An iteration object, created using sltestiteration.

  2. An iteration setting, set using setTestParam.

  3. The iteration registration, added using addIteration.

For example, this script creates an iteration that runs one signal group from a Signal Builder block.

%% Iterate Using a Signal Builder Group
 
% Set up a new iteration object
testItr = sltestiteration;

% Set iteration setting using Signal Builder group
setTestParam(testItr,'SignalBuilderGroup',...
   sltest_signalBuilderGroups{1});

% Add the iteration to run in this test case
% The predefined sltest_testCase variable is used here
addIteration(sltest_testCase,testItr);

For more information about the test iteration class, see sltest.testmanager.TestIteration. You can iterate over multiple items, such as Signal Builder groups. You can iterate over all Signal Builder groups in the block by putting the basic iteration script in a loop:

%% Iterate Over All Signal Builder Groups
 
% Determine the number of possible iterations
numSteps = length(sltest_signalBuilderGroups);
 
% Create each iteration
for k = 1 : numSteps
    % Set up a new iteration object
    testItr = sltestiteration;
 
    % Set iteration settings
    setTestParam(testItr,'SignalBuilderGroup',sltest_signalBuilderGroups{k});
 
    % Add the iteration to run in this test case
    % You can pass in an optional iteration name
    addIteration(sltest_testCase,testItr);
end

Predefined Variables

You can use predefined variables to write iterations scripts. To see the list of predefined variables in the Test Manager, expand the Help on creating test iterations section. You write the iterations script in the script box within the Scripted Iterations section. The script box is a functional workspace, which means the MATLAB® base workspace cannot access information from the script box. If you define variables in the script box, then other workspaces cannot use the variable.

The predefined variables are:

  • sltest_bdroot — Model simulated by the test case, defined as a string

  • sltest_sut — The System Under Test, defined as a string

  • sltest_isharness — true if sltest_bdroot is a harness model, defined as a logical

  • sltest_externalInputs — Name of external inputs, defined as a cell array of strings

  • sltest_parameterSets — Name of parameter override sets, defined as a cell array of strings

  • sltest_configSets — Name of configuration settings, defined as a cell array of strings

  • sltest_tableIterations — Iteration objects created in the iterations table, defined as a cell array of sltest.testmanager.TestIteration objects

  • sltest_testCase — Current test case object, defined as an sltest.testmanager.TestCase object

Scripted Iteration Templates

You can quickly generate iterations for your test case using templates for Signal Builder groups, parameter sets, external inputs, configuration sets, and baseline sets, if you are using a baseline test case. Scripted iteration templates follow lockstep ordering and pairing of test settings. For more information about lockstep ordering, see Create Table Iterations.

For example, if you want to run all signal builder groups in a scripted iteration:

  1. Click Iteration Templates.

  2. Select the test case settings you want to iterate through. Click OK.

    The script is generated and added to the script box below any existing scripts.

  3. To generate a table that gives a preview of the iterations that execute when you run the test case, click Show Iterations.

Capture Baseline Data from Iterations

This example uses:

This example shows how to create a baseline test by capturing data from a test case with table iterations. You create the iterations from Signal Editor scenarios in the model. Before running the example, navigate to a writable folder on the MATLAB® path.

1. Open the model. At the command line, enter

Model = 'sltestCar';
open_system(Model);

2. Open the Test Manager and create a test file. At the command line, enter sltestmgr

3. In the Test Manager, click Test File from Model from the New dropdown.

4. Specify the test file.

  1. Enter sltestCar as the Model.

  2. Enter the test file name or full path in Location.

  3. Select Baseline as the Test Type.

5. Select the test case. Expand the test file and select the sltestCar/Inputs test case.

6. Select the signals for the baseline data:

  1. In the Simulation Outputs section, click Add.

  2. In the model canvas, select the output torque signal and in the Connect dialog, check the box for that signal. Select the vehicle speed signal and check its box in the dialog.

  3. In the Test Manager message dialog box, click Done.

  4. The signals appear in the Logged Signals table.

7. View iterations for the test case:

Expand the Iterations and Table Iterations sections. The iterations for the selected test case automatically appear. The iterations correspond to the four Signal Editor scenarios.

8. Capture baseline data for the iterations:

  1. In the Baseline Criteria section, click the arrow next to Capture.

  2. Select MAT as the File format.

  3. Specify the location to save the baseline data files in the File field.

  4. Select Capture Baselines for Iterations.

  5. Click Capture.

The model simulates for all Signal Editor scenarios. The baseline data for output_torque and vehicle_speed are captured in four MAT files. Also, each baseline data set is added to its corresponding iterations in the table.

Sweep Through a Set of Parameters

Scripted iterations can be used to test a model by sweeping through a set of parameters. You can use this script to try different values for the model workspace parameter Iei and model parameter UpperSaturationLimit in the model sltestCar. Add the script under Iterations > Scripted Iterations.

%% Iterate over Iei parameter

% Set up the parameter values to sweep over
IeiValues = [0.021,0.022,0.022,0.023];
UprSatValues = [2000,3000,4000,5000];
numSteps = length(IeiValues);

% Create each iteration
for k = 1 : numSteps
    % Set up a new iteration object
    testItr = sltestiteration;
    
    % Set value of lei (parameter in model workspace)
    setVariable(testItr,'Name','Iei','Source','model workspace',...
                'Value',IeiValues(k));

    % Set value of UpperSaturationLimit model parameter
    testItr.setModelParam('sltestCar/Engine/Integrator',...
                 'UpperSaturationLimit',UprSatValues(k));
 
    % Add the iteration to run in this test case
    addIteration(sltest_testCase,testItr); 
end

After you add the script, click Show Iterations. You can see the iterations that the script created.

Details of iterations, including parameter names and values, and workspace and model parameter overrides.

Running the test generates a result for each iteration.

Iteration results hierarchy

See Also

|

Related Topics

Simulink Test Documentation

Support