runtests

Run set of tests

collapse all in page

Syntax

results = runtests

results = runtests(tests)

results = runtests(tests,Name,Value)

Description

results = runtests runs all the tests in your current folder, and returns the results as a TestResult object.

example

results = runtests(tests) runs a specified set of tests.

example

results = runtests(tests,Name,Value) runs a set of tests with additional options specified by one or more Name,Value pair arguments.

Examples

collapse all

Run Tests in Working Folder

Open Script

Create a folder myExample in your current working folder, and change into that folder.

In the myExample folder, create a test script, typeTest.m.

%% Test double class
exp = 'double';
act = ones;
assert(isa(act,exp))

%% Test single class
exp = 'single';
act = ones('single');
assert(isa(act,exp))

%% Test uint16 class
exp = 'uint16';
act = ones('uint16');
assert(isa(act,exp))

In the myExample folder, create a test script, sizeValueTest.m.

%% Test size
exp = [7 13];
act = ones([7 13]);
assert(isequal(size(act),exp))

%% Test values
act = ones(42);
assert(unique(act) == 1)

Run all tests in the current folder.

runtests

Running sizeValueTest
..
Done sizeValueTest
__________

Running typeTest
...
Done typeTest
__________


ans = 

  1x5 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   5 Passed, 0 Failed, 0 Incomplete.
   0.038077 seconds testing time.

MATLAB® ran 5 tests. There are 2 passing tests from sizeValueTest and 3 passing tests from typeTest.

Run Tests Using File Name

Open Script

Create the test file shown below, and save it as runtestsExampleTest.m on your MATLAB path.

function tests = runtestsExampleTest
tests = functiontests(localfunctions);

function testFunctionOne(testCase)

Run the tests.

results = runtests('runtestsExampleTest.m');

Running runtestsExampleTest
.
Done runtestsExampleTest
__________

Run Tests in Subfolder

If it does not exist, create the test file, runtestsExampleTest.m, in the example above.

Create a subfolder, tmpTest, and, in that folder, create the following runtestsExampleSubFolderTest.m file.

function tests = runtestsExampleSubFolderTest
tests = functiontests(localfunctions);

function testFunctionTwo(testCase)

Run the tests from the folder above tmpTest by setting 'IncludeSubfolders' to true.

results = runtests(pwd,'IncludeSubfolders',true);

Running runtestsExampleTest
.
Done runtestsExampleTest
__________

Running runtestsExampleSubFolderTest
.
Done runtestsExampleSubFolderTest
__________

runtests ran the tests in both the current folder and the subfolder.

If you do not specify the 'IncludeSubfolders' property for the runtests function, it does not run the test in the subfolder.

results = runtests(pwd);

Running runtestsExampleTest
.
Done runtestsExampleTest
__________

Run Tests in Parallel

Create the following test file, and save it as runInParallelTest.m on your MATLAB^® path.

function tests = runInParallelTest
tests = functiontests(localfunctions);

function testA(testCase)
verifyEqual(testCase,5,5);

function testB(testCase)
verifyTrue(testCase,logical(1));

function testC(testCase)
verifySubstring(testCase,'SomeLongText','Long');

function testD(testCase)
verifySize(testCase,ones(2,5,3),[2 5 3]);

function testE(testCase)
verifyGreaterThan(testCase,3,2);

function testF(testCase)
verifyEmpty(testCase,{},'Cell array is not empty.');

function testG(testCase)
verifyMatches(testCase,'Some Text','Some [Tt]ext');

Run the tests in parallel. Running tests in parallel requires Parallel Computing Toolbox™. The testing framework might vary the order and number of groups or which tests it includes in each group.

results = runtests('runInParallelTest','UseParallel',true);

Split tests into 7 groups and running them on 4 workers.
----------------
Finished Group 2
----------------
Running runInParallelTest
.
Done runInParallelTest
__________


----------------
Finished Group 3
----------------
Running runInParallelTest
.
Done runInParallelTest
__________


----------------
Finished Group 1
----------------
Running runInParallelTest
.
Done runInParallelTest
__________


----------------
Finished Group 4
----------------
Running runInParallelTest
.
Done runInParallelTest
__________


----------------
Finished Group 6
----------------
Running runInParallelTest
.
Done runInParallelTest
__________


----------------
Finished Group 5
----------------
Running runInParallelTest
.
Done runInParallelTest
__________


----------------
Finished Group 7
----------------
Running runInParallelTest
.
Done runInParallelTest
__________

Run Select Parameterized Tests

Open Script

In your working folder, create testZeros.m. This class contains four test methods.

classdef testZeros < matlab.unittest.TestCase
    properties (TestParameter)
        type = {'single','double','uint16'};
        outSize = struct('s2d',[3 3], 's3d',[2 5 4]);
    end
    
    methods (Test)
        function testClass(testCase, type, outSize)
            testCase.verifyClass(zeros(outSize,type), type);
        end
        
        function testSize(testCase, outSize)
            testCase.verifySize(zeros(outSize), outSize);
        end
        
        function testDefaultClass(testCase)
            testCase.verifyClass(zeros, 'double');
        end
        function testDefaultSize(testCase)
            testCase.verifySize(zeros, [1 1]);
        end
        
        function testDefaultValue(testCase)
            testCase.verifyEqual(zeros,0);
        end
    end
end

The full test suite has 11 test elements: 6 from the testClass method, 2 from the testSize method, and 1 each from the testDefaultClass, testDefaultSize, and testDefaultValue methods.

At the command prompt, run all the parameterizations for the testSize method.

runtests('testZeros/testSize')

Running testZeros
..
Done testZeros
__________


ans = 

  1x2 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   2 Passed, 0 Failed, 0 Incomplete.
   0.041371 seconds testing time.

The runtests function executed the two parameterized tests from the testSize method. Alternatively, you can specify the test procedure name with runtests('testZeros','ProcedureName','testSize').

Run the test elements that use the outSize parameter property.

runtests('testZeros','ParameterProperty','outSize')

Running testZeros
........
Done testZeros
__________


ans = 

  1x8 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   8 Passed, 0 Failed, 0 Incomplete.
   0.059447 seconds testing time.

The runtests function executed eight tests that use the outSize parameter property: six from the testClass method and two from the testSize method.

Run the test elements that use the single parameter name.

runtests('testZeros','ParameterName','single')

Running testZeros
..
Done testZeros
__________


ans = 

  1x2 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   2 Passed, 0 Failed, 0 Incomplete.
   0.010709 seconds testing time.

The runtests function executed the two tests from the testClass method that use the outSize parameter name.

Input Arguments

collapse all

`tests` — Array of tests
character vector | cell array of character vectors

Suite of tests specified as a string array, character vector, or cell array of character vectors. Each character vector in the cell array can contain the name of a test file, a test class, a test suite element name, a package containing your test classes, a folder containing your test files, or a project folder containing test files.

Example: runtests('ATestFile.m')

Example: runtests('ATestFile/aTest')

Example: runtests('mypackage.MyTestClass')

Example: runtests(pwd)

Example: runtests({'mypackage.MyTestClass','ATestFile.m',pwd,'mypackage.subpackage'})

Example: runtests('C:/projects/project1/')

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: runtests(tests,'Name','productA_*') runs test elements with a name that starts with 'productA_'.

`'BaseFolder'` — Name of base folder
string array | character vector | cell array of character vectors

Name of the base folder that contains the file defining the test class, function, or script, specified as a string array, character vector, or cell array of character vectors. This argument filters TestSuite array elements. For the testing framework to include a test in the suite, the Test element must be contained in one of the base folders specified by BaseFolder. If none of the Test elements matches a base folder, an empty test suite is returned. Use the wildcard character * to match any number of characters. Use the question mark character ? to match a single character. For test files defined in packages, the base folder is the parent of the top-level package folder.

`'Debug'` — Indicator to apply debugging capabilities
`false` (default) | `true` | `0` | `1`

Indicator to apply debugging capabilities when running tests, specified as false or true (0 or 1). For example, if a test failure is encountered, the framework pauses test execution to enter debug mode.

`'IncludeSubfolders'` — Indicator to run tests in subfolders
`false` (default) | `true` | `0` | `1`

Indicator to run tests in subfolders, specified as false or true (0 or 1). By default the framework runs tests in the specified folders, but not in their subfolders.

`'IncludeSubpackages'` — Indicator to run tests in subpackages
`false` (default) | `true` | `0` | `1`

Indicator to run tests in subpackages, specified as false or true (0 or 1). By default the framework runs tests in the specified packages, but not in their subpackages.

`'IncludeReferencedProjects'` — Indicator to include tests from referenced projects
`false` (default) | `true` | `0` | `1`

Indicator to include tests from referenced projects, specified as logical false or true. For more information on referenced projects, see Componentize Large Projects.

`'LoggingLevel'` — Maximum verbosity level for logged diagnostics
1 (default) | 0 | 2 | 3 | 4 | `matlab.unittest.Verbosity` enumeration

Maximum verbosity level for logged diagnostics included for the test run, specified as an integer value from 0 through 4, or as a matlab.unittest.Verbosity enumeration object. The runtests function includes diagnostics that are logged at this level and below. Integer values correspond to the members of the matlab.unittest.Verbosity enumeration.

By default runtests includes diagnostics logged at the matlab.unittest.Verbosity.Terse level (level 1). To exclude logged diagnostics, specify LoggingLevel as Verbosity.None (level 0).

Logged diagnostics are diagnostics that you supply to the testing framework with a call to the log (TestCase) or log (Fixture) method.

Numeric Representation	Enumeration Member Name	Verbosity Description
`0`	`None`	No information
`1`	`Terse`	Minimal information
`2`	`Concise`	Moderate amount of information
`3`	`Detailed`	Some supplemental information
`4`	`Verbose`	Lots of supplemental information

`'OutputDetail'` — Display level for event details
0 | 1 | 2 | 3 | 4 | `matlab.unittest.Verbosity` enumeration

Display level for event details, specified as an integer value from 0 through 4, or as a matlab.unittest.Verbosity enumeration object. Integer values correspond to the members of the matlab.unittest.Verbosity enumeration.

The runtests function displays failing and logged events with the amount of detail specified by OutputDetail. By default, runtests displays failing and logged events at the matlab.unittest.Verbosity.Detailed level (level 3) and test run progress at the matlab.unittest.Verbosity.Concise level (level 2).

Numeric Representation	Enumeration Member Name	Verbosity Description
`0`	`None`	No information
`1`	`Terse`	Minimal information
`2`	`Concise`	Moderate amount of information
`3`	`Detailed`	Some supplemental information
`4`	`Verbose`	Lots of supplemental information

`'Name'` — Name of suite element
string array | character vector | cell array of character vectors

Name of the suite element, specified as a string array, character vector, or cell array of character vectors. This argument filters TestSuite array elements. For the testing framework to include a test in the suite, the Name property of the Test element must match one of the names specified by Name. If none of the Test elements has a matching name, an empty test suite is returned. Use the wildcard character * to match any number of characters. Use the question mark character ? to match a single character.

`'ParameterProperty'` — Name of parameterization property
string array | character vector | cell array of character vectors

Name of a test class property that defines a parameter used by the test suite element, specified as a string array, character vector, or cell array of character vectors. This argument filters TestSuite array elements. For the testing framework to include a test in the suite, the Parameterization property of the Test element must contain at least one of the property names specified by ParameterProperty. If none of the Test elements has a matching property name, an empty test suite is returned. Use the wildcard character * to match any number of characters. Use the question mark character ? to match to a single character.

`'ParameterName'` — Name of parameter
string array | character vector | cell array of character vectors

Name of a parameter used by the test suite element, specified as a string array, character vector, or cell array of character vectors. MATLAB generates parameter names based on the test class property that defines the parameters:

If the property value is a cell array of character vectors, MATLAB generates parameter names from the values in the cell array. Otherwise, MATLAB specifies parameter names as value1, value2, …, valueN.
If the property value is a structure, MATLAB generates parameter names from the structure fields.

The ParameterName argument filters TestSuite array elements. For the testing framework to include a test in the suite, the Parameterization property of the Test element must contain at least one of the parameter names specified by ParameterName. If none of the Test elements has a matching parameter name, an empty test suite is returned. Use the wildcard character * to match any number of characters. Use the question mark character ? to match a single character.

`'ProcedureName'` — Name of test procedure
string array | character vector | cell array of character vectors

Name of the test procedure, specified as a string array, character vector, or cell array of character vectors. This argument filters TestSuite array elements. For the testing framework to include a test in the suite, the ProcedureName property of the Test element must match one of the procedure names specified by ProcedureName. If none of the Test elements has a matching procedure name, an empty test suite is returned. Use the wildcard character * to match any number of characters. Use the question mark character ? to match a single character.

In a class-based test, the ProcedureName is the name of the test method. In a function-based test, it is the name of the local function that contains the test. In a script-based test, it is a name generated from the test section title. Unlike Name, the name of the test procedure does not include any class or package name or information about parameterization.

`'ReportCoverageFor'` — Path to source code to include in code coverage report
path to file or folder

Path to source code to include in code coverage report, specified as a string array, character vector, or cell array of character vectors. Using this option with runtests runs the specified tests and produces a code coverage report for the specified code files. The report shows which lines in the source code were executed by the tests.

The source code can be an absolute or relative path to one or more folders or to files that have a .m, .mlx, or .mlapp extension.

Example: runtests(tests,'ReportCoverageFor','mySource.m')

Data Types: char | string | cell

`'Strict'` — Indicator to apply strict checks
`false` (default) | `true` | `0` | `1`

Indicator to apply strict checks when running tests, specified as false or true (0 or 1). For example, the framework generates a qualification failure if a test issues a warning.

`'Superclass'` — Name of class that test class derives from
string array | character vector | cell array of character vectors

Name of the class that the test class derives from, specified as a string array, character vector, or cell array of character vectors. This argument filters TestSuite array elements. For the testing framework to include a test in the suite, the TestClass property of the Test element must point to a test class that derives from one of the classes specified by Superclass. If none of the Test elements matches a class, an empty test suite is returned.

`'Tag'` — Name of test element tag
string array | character vector | cell array of character vectors

Name of a test tag used by the test suite element, specified as a string array, character vector, or cell array of character vectors. This argument filters TestSuite array elements. For the testing framework to include a test in the suite, the Tags property of the Test element must contain at least one of the tag names specified by Tag. If none of the Test elements has a matching tag name, an empty test suite is returned. Use the wildcard character * to match any number of characters. Use the question mark character ? to match a single character.

`'UseParallel'` — Indicator to run tests in parallel
`false` (default) | `true` | `0` | `1`

Indicator to run tests in parallel, specified as false or true (0 or 1).

By default runtests runs tests in serial. If you set UseParallel to true, then runtests divides the test suite into separate groups and runs the groups in parallel if:

Parallel Computing Toolbox is installed.
An open parallel pool exists or automatic pool creation is enabled in the Parallel Preferences.

Otherwise, runtests runs tests in serial regardless of the value for UseParallel.

Testing in parallel might not be compatible with other options. For example, testing occurs in serial if UseParallel and Debug are both set to true. When running in parallel, the testing framework might vary the order and number of groups or which tests it includes in each group.

`'GenerateBaselines'` — Indicator to create or update baseline data
`false` (default) | `true` | `0` | `1`

Indicator to create or update a MAT-file being used in a test with specific qualification methods, specified as false or true (0 or 1) . You must have a Simulink^® Test™ license to use GenerateBaselines.

When you specify this argument as true, your test must use at least one of these qualification methods of the sltest.TestCase (Simulink Test) class:

verifySignalsMatch (for example, testCase.verifySignalsMatch(actVal,'myBaseline.mat'))
assumeSignalsMatch
assertSignalsMatch
fatalAssertSignalsMatch

For more information, see Using MATLAB-Based Simulink Tests in the Test Manager (Simulink Test).

Tips

When you use shared test fixtures in your tests and specify the input to the runtests function as a string array or cell array of character vectors, the testing framework sorts the array to reduce shared fixture setup and teardown operations. As a result, the tests might run in an order that is different from the order of elements in the input array. For more information, see sortByFixtures.
Starting in R2020b, you can create standalone applications that support running tests in parallel (requires MATLAB Compiler™ and Parallel Computing Toolbox). Use the directive %#function parallel.Pool in your code so that MATLAB Compiler can locate and package all of the components required for running tests in parallel. For more information, see Compile MATLAB Unit Tests.

Extended Capabilities

Automatic Parallel Support
Accelerate code by automatically running computation in parallel using Parallel Computing Toolbox™.

To run in parallel, set the 'UseParallel' option to true.

For more general information about parallel computing, see Run MATLAB Functions with Automatic Parallel Support (Parallel Computing Toolbox).

Documentation

runtests

Syntax

Description

Examples

Run Tests in Working Folder

Run Tests Using File Name

Run Tests in Subfolder

Run Tests in Parallel

Run Select Parameterized Tests

Input Arguments

`tests` — Array of tests
character vector | cell array of character vectors

Name-Value Pair Arguments

`'BaseFolder'` — Name of base folder
string array | character vector | cell array of character vectors

`'Debug'` — Indicator to apply debugging capabilities
`false` (default) | `true` | `0` | `1`

`'IncludeSubfolders'` — Indicator to run tests in subfolders
`false` (default) | `true` | `0` | `1`

`'IncludeSubpackages'` — Indicator to run tests in subpackages
`false` (default) | `true` | `0` | `1`

`'IncludeReferencedProjects'` — Indicator to include tests from referenced projects
`false` (default) | `true` | `0` | `1`

`'LoggingLevel'` — Maximum verbosity level for logged diagnostics
1 (default) | 0 | 2 | 3 | 4 | `matlab.unittest.Verbosity` enumeration

`'OutputDetail'` — Display level for event details
0 | 1 | 2 | 3 | 4 | `matlab.unittest.Verbosity` enumeration

`'Name'` — Name of suite element
string array | character vector | cell array of character vectors

`'ParameterProperty'` — Name of parameterization property
string array | character vector | cell array of character vectors

`'ParameterName'` — Name of parameter
string array | character vector | cell array of character vectors

`'ProcedureName'` — Name of test procedure
string array | character vector | cell array of character vectors

`'ReportCoverageFor'` — Path to source code to include in code coverage report
path to file or folder

`'Strict'` — Indicator to apply strict checks
`false` (default) | `true` | `0` | `1`

`'Superclass'` — Name of class that test class derives from
string array | character vector | cell array of character vectors

`'Tag'` — Name of test element tag
string array | character vector | cell array of character vectors

`'UseParallel'` — Indicator to run tests in parallel
`false` (default) | `true` | `0` | `1`

`'GenerateBaselines'` — Indicator to create or update baseline data
`false` (default) | `true` | `0` | `1`

Tips

Extended Capabilities

Automatic Parallel Support
Accelerate code by automatically running computation in parallel using Parallel Computing Toolbox™.

See Also

Topics

MATLAB Documentation

Support

Documentation

runtests

Syntax

Description

Examples

Run Tests in Working Folder

Run Tests Using File Name

Run Tests in Subfolder

Run Tests in Parallel

Run Select Parameterized Tests

Input Arguments

tests — Array of tests character vector | cell array of character vectors

Name-Value Pair Arguments

'BaseFolder' — Name of base folder string array | character vector | cell array of character vectors

'Debug' — Indicator to apply debugging capabilities false (default) | true | 0 | 1

'IncludeSubfolders' — Indicator to run tests in subfolders false (default) | true | 0 | 1

'IncludeSubpackages' — Indicator to run tests in subpackages false (default) | true | 0 | 1

'IncludeReferencedProjects' — Indicator to include tests from referenced projects false (default) | true | 0 | 1

'LoggingLevel' — Maximum verbosity level for logged diagnostics 1 (default) | 0 | 2 | 3 | 4 | matlab.unittest.Verbosity enumeration

'OutputDetail' — Display level for event details 0 | 1 | 2 | 3 | 4 | matlab.unittest.Verbosity enumeration

'Name' — Name of suite element string array | character vector | cell array of character vectors

'ParameterProperty' — Name of parameterization property string array | character vector | cell array of character vectors

'ParameterName' — Name of parameter string array | character vector | cell array of character vectors

'ProcedureName' — Name of test procedure string array | character vector | cell array of character vectors

'ReportCoverageFor' — Path to source code to include in code coverage report path to file or folder

'Strict' — Indicator to apply strict checks false (default) | true | 0 | 1

'Superclass' — Name of class that test class derives from string array | character vector | cell array of character vectors

'Tag' — Name of test element tag string array | character vector | cell array of character vectors

'UseParallel' — Indicator to run tests in parallel false (default) | true | 0 | 1

'GenerateBaselines' — Indicator to create or update baseline data false (default) | true | 0 | 1

Tips

Extended Capabilities

Automatic Parallel Support Accelerate code by automatically running computation in parallel using Parallel Computing Toolbox™.

See Also

Topics

MATLAB Documentation

Support

`tests` — Array of tests
character vector | cell array of character vectors

`'BaseFolder'` — Name of base folder
string array | character vector | cell array of character vectors

`'Debug'` — Indicator to apply debugging capabilities
`false` (default) | `true` | `0` | `1`

`'IncludeSubfolders'` — Indicator to run tests in subfolders
`false` (default) | `true` | `0` | `1`

`'IncludeSubpackages'` — Indicator to run tests in subpackages
`false` (default) | `true` | `0` | `1`

`'IncludeReferencedProjects'` — Indicator to include tests from referenced projects
`false` (default) | `true` | `0` | `1`

`'LoggingLevel'` — Maximum verbosity level for logged diagnostics
1 (default) | 0 | 2 | 3 | 4 | `matlab.unittest.Verbosity` enumeration

`'OutputDetail'` — Display level for event details
0 | 1 | 2 | 3 | 4 | `matlab.unittest.Verbosity` enumeration

`'Name'` — Name of suite element
string array | character vector | cell array of character vectors

`'ParameterProperty'` — Name of parameterization property
string array | character vector | cell array of character vectors

`'ParameterName'` — Name of parameter
string array | character vector | cell array of character vectors

`'ProcedureName'` — Name of test procedure
string array | character vector | cell array of character vectors

`'ReportCoverageFor'` — Path to source code to include in code coverage report
path to file or folder

`'Strict'` — Indicator to apply strict checks
`false` (default) | `true` | `0` | `1`

`'Superclass'` — Name of class that test class derives from
string array | character vector | cell array of character vectors

`'Tag'` — Name of test element tag
string array | character vector | cell array of character vectors

`'UseParallel'` — Indicator to run tests in parallel
`false` (default) | `true` | `0` | `1`

`'GenerateBaselines'` — Indicator to create or update baseline data
`false` (default) | `true` | `0` | `1`

Automatic Parallel Support
Accelerate code by automatically running computation in parallel using Parallel Computing Toolbox™.