Coverage Filters

During the verification process, a model can contain several constructs that prevent full model coverage, such as a subsystem that contains a driver for a controller that is not tested and is not relevant to the validation process. You can exclude this subsystem from the coverage results.

Alternatively, you may have testing criteria that requires exercising certain aspects of a block, such as hitting particular decision points. If it is not feasible to satisfy all objective outcomes for this block, and you did not intend for your tests to exercise these unsatisfied outcomes, then you could justify this missing coverage.

Filtering these constructs in coverage results by excluding or justifying them allows you to focus on other aspects of missing coverage that can and should be tested.

Coverage filters are stored in .cvf files. Each filter consists of rules that exclude or justify certain model objects, or individual coverage objective outcomes. Multiple filter files can be applied to coverage results for a model. Furthermore, multiple models can also make use of the same filter file.

Coverage filters can be created and applied either before or after simulating a model. Both workflows are described in this example.

Open Model

This example makes use of the slvnvdemo_covfilt model. The model shows some common patterns that might need to be filtered from coverage results. Open the model.

open_system('slvnvdemo_covfilt');

Specify Items to Exclude from Coverage Results before Simulation

The library block slvnvdemo_covfilt_lib/protected division protects against division by zero. If you determine that your testing is not expected or intended to fully cover every instance of this block in this context, the block can be excluded from coverage results.

In the Simulink Editor, right-click an instance of the protected division library block and navigate to the Coverage options. The options for this block allow you to filter the specific instance of the library or all references to this library. Select Exclude referenced library: slvnvdemo_covfilt_lib/protected division to filter all references.

This opens the Filter Editor section of the Coverage Results Explorer. Note that a new filter file, initially named Untitled, has been created, and the filter rule for excluding all references to the library block has been added.

Specify a Name and Description for the new filter file. In the table, open the Rationale field for the new rule and enter text describing why this block is excluded, such as "division by zero protection". Click Apply when finished to save the filter file. A file dialog will prompt you to specify where to save this file.

Reuse Existing Filter File

You can share and reapply filter files with generalized rules to filter coverage results for various different models that contain similar constructs.

For example, the existing filter file Filter_Tick.cvf captures a rule to exclude the Stateflow temporal event tick from coverage results. This event can never be false and, therefore, could prevent full Condition and MCDC coverage in any model using tick in event-based temporal logic in Stateflow.

Because slvnvdemo_covfilt/Mode Logic contains such a construct, you can apply the filter file Filter_Tick.cvf to the model.

To apply this existing filter file, right-click on the Applied filters node in the Coverage Results Explorer and select Load filter. In the file dialog that opens, select Filter_Tick.cvf and click Open.

Note that Applied filters now lists both Filter_DivBy0 and Filter_Tick.

Simulate and Review Filtered Coverage Results

Click the Run (Coverage) button to simulate the model and record coverage. When the simulation completes, coverage results are highlighted on the model and the Coverage Details window is opened.

Notice that both references to the protected division library block are colored gray in the Simulink canvas, indicating that they have been excluded from the coverage results.

Scroll through the content in the Coverage Details window, and note the section titled Objects Filtered from Coverage Analysis. This section lists each of the excluded elements and the corresponding rationales for each and is organized by filter file. Both Filter_DivBy0 and Filter_Tick have been applied.

Create a New Filter File

The filter files used above capture generalized filter rules that could be broadly applicable to many different models.

Create another filter file to separately capture filter rules exclusively relevant to this model.

In the Coverage Results Explorer, right-click on the Applied filters node and select New filter.

Enter a name and description for this filter file. Click Apply and specify where to save the file.

Exclude Items from Coverage Results after Simulation

The previous sections of this example show how to specify filter rules to apply before running a simulation. You can also create and apply filter rules to coverage results after simulation. This allows you to review coverage results, create or adjust filters, and generate a new coverage report without having to rerun the simulation.

Consider, for example, the Switchable config subsystem. It is a common design pattern to use constant values to drive subsystem enable ports for changing model configurations; however, the enable logic and subsystem contents might lead to missing coverage. This configuration is not used in this model, and therefore can be excluded from coverage results.

In the Simulink Editor, click on the Switchable config subsystem. The Coverage Details window will navigate to the details for this subsystem. Under these details, click the Justify or Exclude link.

A new filter rule is added to the currently selected filter file in the Filter Editor. By default, the mode for this rule is set to "Excluded". Enter the rationale for this rule, such as "unused config".

Click Apply when finished. This saves the changes to the filter file and automatically updates the coverage results displayed on the model. Notice that the Switchable config subsystem is now shown as gray, indicating that it has been excluded from the coverage results.

Justify Individual Objective Outcome from the Coverage Results

In the Simulink Editor, click on the Saturation block and review the coverage results in the Coverage Details window. Notice that two Decision outcomes are unsatisfied. This is because the Saturation block has a lower limit of 0 and an upper limit of 200. However, the input to this block is the rate signal, which can never be less than or equal to 0. As such, the lower limit of the Saturation block is not expected to be fully exercised, so the corresponding Decision outcome can be justified.

Click on the Add justification rule icon next to the false outcome for Decision "input > lower limit".

A new filter rule is added to the currently selected filter file in the Filter Editor. Specify a justification rationale, such as "rate > 0".

Click Apply when finished to save the filter file and update the coverage results shown on the model.

Notice that in the Coverage Details window, the Saturation block's justified outcome is highlighted in light blue and has a link to the rationale. The true outcome of the decision "input >= upper limit" is not filtered and is reported as unsatisfied. As such, the Saturation block is still highlighted red in the model to indicate missing coverage.

Automatically Generate Filter Rules for Dead Logic

In some cases, missing coverage is due to dead logic. The associated coverage objectives are unsatisfiable. If this logic is meant for constructs that you do not wish to remove from your model, then those coverage objectives can be justified.

If you have a Simulink Design Verifier™ license, then you can automatically create justification filter rules for dead logic.

In the Coverage Results Explorer, select the Applied filters node. In the Filter Editor pane, select the option Make justification filter rules for dead logic (using Simulink Design Verifier).

This option uses Simulink Design Verifier™ to analyze the model for dead logic. Upon completion, a new filter file is created, and justification rules are added for each of the corresponding coverage outcomes.

Specify a name and description for this filter file. Click Apply when finished. In the resulting file dialog, specify where to save this filter file.

Close the Simulink Design Verifier Results windows that opened.

Review Filtered Coverage Results

In the Simulink Editor, navigate to the Review Results section of the Coverage app, and click Coverage Highlighting.

After applying all four filters used in this example, the simulation now achieves 100% Condition, 63% Decision, and 75% Execution coverage for this model. Note that the coverage results no longer contain any model objects that receive MCDC, so this metric is not listed.

The remaining missing coverage in the Mode Logic chart, time capture subsystem, and Saturation block indicates inadequate testing and should be addressed by using further simulation with input values that more thoroughly exercise these elements.

Conclusion

Coverage filtering provides various methods for indicating aspects of your model that are not expected or intended to be fully exercised in the current testing context.

Filtering model objects and coverage objective outcomes that do not need to be exercised allows you to focus on the missing coverage for constructs that can and should be tested.