Run Tests with Customized Test Runner

This example shows how to create a test suite and customize the test runner to report on test run progress and produce CI-compatible artifacts.

In a file in your current folder, create the function quadraticSolver, which returns the roots of quadratic polynomials.

function roots = quadraticSolver(a,b,c)
% quadraticSolver returns solutions to the
% quadratic equation a*x^2 + b*x + c = 0.

if ~isa(a,'numeric') || ~isa(b,'numeric') || ~isa(c,'numeric')
    error('quadraticSolver:InputMustBeNumeric', ...
        'Coefficients must be numeric.');
end

roots(1) = (-b + sqrt(b^2 - 4*a*c)) / (2*a);
roots(2) = (-b - sqrt(b^2 - 4*a*c)) / (2*a);

end

To test quadraticSolver, create the test class SolverTest in your current folder.

classdef SolverTest < matlab.unittest.TestCase
    methods(Test)
        function testRealSolution(testCase)
            actSolution = quadraticSolver(1,-3,2);
            expSolution = [2,1];
            testCase.verifyEqual(actSolution,expSolution)
        end
        function testImaginarySolution(testCase)
            actSolution = quadraticSolver(1,2,10);
            expSolution = [-1+3i, -1-3i];
            testCase.verifyEqual(actSolution,expSolution)
        end
        function testNonNumericInput(testCase)
            testCase.verifyError(@()quadraticSolver(1,'-3',2), ...
                'quadraticSolver:InputMustBeNumeric')
        end
    end
end

At the command prompt, create a test suite from the SolverTest class.

suite = testsuite('SolverTest');

Create a TestRunner instance that produces output using the matlab.unittest.TestRunner.withTextOutput method. This method enables you to set the maximum verbosity level for logged diagnostics and the display level for test event details. In this example, the test runner displays test run progress at the matlab.unittest.Verbosity.Detailed level (level 3).

import matlab.unittest.TestRunner
runner = TestRunner.withTextOutput('OutputDetail',3);

Create a TestReportPlugin instance that sends output to the file testreport.pdf and add the plugin to the test runner.

import matlab.unittest.plugins.TestReportPlugin
pdfFile = 'testreport.pdf';
p1 = TestReportPlugin.producingPDF(pdfFile);
runner.addPlugin(p1)

Create an XMLPlugin instance that writes JUnit-style XML output to the file junittestresults.xml. Then, add the plugin to the test runner.

import matlab.unittest.plugins.XMLPlugin
xmlFile = 'junittestresults.xml';
p2 = XMLPlugin.producingJUnitFormat(xmlFile);
runner.addPlugin(p2)

Create a plugin that outputs a Cobertura code coverage report for the source code in the file quadraticSolver.m. Instruct the plugin to write its output to the file cobertura.xml and add the plugin to the test runner.

import matlab.unittest.plugins.CodeCoveragePlugin
import matlab.unittest.plugins.codecoverage.CoberturaFormat
sourceCodeFile = 'quadraticSolver.m';
reportFile = 'cobertura.xml';
reportFormat = CoberturaFormat(reportFile);
p3 = CodeCoveragePlugin.forFile(sourceCodeFile,'Producing',reportFormat);
runner.addPlugin(p3)

Run the tests.

results = runner.run(suite)

 Running SolverTest
  Setting up SolverTest
  Done setting up SolverTest in 0 seconds
   Running SolverTest/testRealSolution
   Done SolverTest/testRealSolution in 0.41095 seconds
   Running SolverTest/testImaginarySolution
   Done SolverTest/testImaginarySolution in 0.016832 seconds
   Running SolverTest/testNonNumericInput
   Done SolverTest/testNonNumericInput in 0.19255 seconds
  Tearing down SolverTest
  Done tearing down SolverTest in 0 seconds
 Done SolverTest in 0.62033 seconds
__________

Generating test report. Please wait.
    Preparing content for the test report.

    Adding content to the test report.
    Writing test report to file.
Test report has been saved to:
 /tmp/BR2020bd_1459859_105924/mlx_to_docbook51/tpe36cedab/matlab-ex57066699/testreport.pdf

results = 
  1x3 TestResult array with properties:

    Name
    Passed
    Failed
    Incomplete
    Duration
    Details

Totals:
   3 Passed, 0 Failed, 0 Incomplete.
   0.62033 seconds testing time.

List the files in your current folder. The three specified artifacts are stored in your current folder.

dir 

.                                                       
..                                                      
GenerateArtifactsUsingMATLABUnitTestPluginsExample.mlx  
SolverTest.m                                            
cobertura.xml                                           
junittestresults.xml                                    
quadraticSolver.m                                       
testreport.pdf                                          

You can process the generated artifacts on CI platforms. You also can view the contents of the generated artifacts. For example, open the PDF test report.

open('testreport.pdf')