onFailure

Class: matlab.unittest.TestCase
Package: matlab.unittest

Dynamically add diagnostics for test failures

expand all in page

Syntax

onFailure(testcase,failureDiag)
onFailure(testcase,failureDiag,'IncludingAssumptionFailures',tf)

Description

example

onFailure(testcase,failureDiag) adds diagnostics for test failures. If a test fails, then the test framework executes the diagnostics. By default, these diagnostics execute upon verification failures, assertion failures, fatal assertion failures, and uncaught exceptions.

onFailure(testcase,failureDiag,'IncludingAssumptionFailures',tf) indicates if the test framework also executes diagnostics upon assumption failures. To execute diagnostics upon assumption failures also, set tf to true.

Input Arguments

expand all

Instance of test case, specified as a matlab.unittest.TestCase.

Diagnostic information to display upon a failure, specified as a character vector, string array, function handle, or array of matlab.unittest.diagnostics.Diagnostic instances.

Example: @() disp('Failure Detected')

Example: matlab.unittest.diagnostics.ScreenshotDiagnostic

React to assumption failures, specified as false (logical 0) or true (logical 1). By default, this value is false and the test framework executes diagnostics upon verification failures, assertion failures, fatal assertion failures, and uncaught exceptions. However, the test framework does not execute diagnostics upon assumption failures. To execute additional diagnostics upon assumption failures, specify this value as true.

Examples

expand all

In your current working folder, create the following test class SampleOnFailureTest.m.

classdef SampleOnFailureTest < matlab.unittest.TestCase
    methods(TestMethodSetup)
        function addFailureDiag(testCase)
            testCase.onFailure('Failure Detected');
        end
    end
    methods (Test)
        function verificationFailTest(testCase)
            testCase.onFailure(@()disp(datetime))
            testCase.verifyEqual(42,13)
        end
        function passingTest(testCase)
            testCase.assertTrue(true)
        end
        function assumptionFailTest(testCase)
            testCase.assumeEmpty(rand(2))
        end
        function assertionFailTest(testCase)
            act = randi(100,1,15);
            floor = randi(100,1,15);
            f = figure; hold on;
            plot(1:length(act),act,1:length(floor),floor);
            legend('actual','floor')
            testCase.addTeardown(@close,f)
            
            import matlab.unittest.diagnostics.FigureDiagnostic
            testCase.onFailure(FigureDiagnostic(f,'Formats','png'))
            
            testCase.assertGreaterThan(act,floor)
        end
    end
end

At the command prompt, run the tests. The SampleOnFailureTest class has these results.

  • The diagnostic message 'Failure Detected' displays for each test with a verification, assertion, or fatal assertion failure because addFailureDiag calls onFailure in a function in the TestMethodSetup block.

  • The verificationFailTest test adds another diagnostic upon failure that displays the current date and time.

  • The assumptionFailTest test fails by assumption. Therefore the 'Failure Detected' message does not display.

  • The assertionFailTest test plots the data. If the test fails, the test framework saves the plot.

results = runtests('SampleOnFailureTest');
Running SampleOnFailureTest

================================================================================
Verification failed in SampleOnFailureTest/verificationFailTest.

    ---------------------
    Framework Diagnostic:
    ---------------------
    verifyEqual failed.
    --> The values are not equal using "isequaln".
    --> Failure table:
            Actual    Expected    Error     RelativeError  
            ______    ________    _____    ________________
        
            42        13          29       2.23076923076923
    
    Actual Value:
        42
    Expected Value:
        13

    ----------------------
    Additional Diagnostic:
    ----------------------
    Failure Detected

    ------------------
    Stack Information:
    ------------------
    In C:\work\SampleOnFailureTest.m (SampleOnFailureTest.verificationFailTest) at 10
================================================================================
..
================================================================================
SampleOnFailureTest/assumptionFailTest was filtered.
Details
================================================================================
.
================================================================================
Assertion failed in SampleOnFailureTest/assertionFailTest and it did not run to completion.

    ---------------------
    Framework Diagnostic:
    ---------------------
    assertGreaterThan failed.
    --> Each element must be greater than each corresponding element of the minimum value array.
        
        Failing Indices:
             1     2     3     4     5     6     9    12    13    14
    
    Actual Value:
        55    68     4    81    75    13    53    33    55    40    42    19    26     3    93
    Minimum Value (Exclusive):
        66    94    17    93    80    58    45    26    76    23     7    77    68    72    65

    ----------------------
    Additional Diagnostic:
    ----------------------
    Failure Detected

    ----------------------
    Additional Diagnostic:
    ----------------------
    Figure saved to:
    --> C:\Temp\070a23db-3903-4abb-9976-2d29ec0a1e38\Figure_c2528bb2-6d72-48d5-a8ed-2a9d5ae7b3ee.png

    ------------------
    Stack Information:
    ------------------
    In C:\work\SampleOnFailureTest.m (SampleOnFailureTest.assertionFailTest) at 28
================================================================================
.
Done SampleOnFailureTest
__________

Failure Summary:

     Name                                      Failed  Incomplete  Reason(s)
    =======================================================================================
     SampleOnFailureTest/verificationFailTest    X                 Failed by verification.
    ---------------------------------------------------------------------------------------
     SampleOnFailureTest/assumptionFailTest                X       Filtered by assumption.
    ---------------------------------------------------------------------------------------
     SampleOnFailureTest/assertionFailTest       X         X       Failed by assertion.

Tips

  • To add a diagnostic for each test in your class, call the onFailure method from a function in the TestMethodSetup block.

See Also

Introduced in R2017b

MATLAB Documentation

Support