Coverage Criteria for Set-Based Specifications

Abstract

Model-based testing (MBT) studies how test cases are generated from a model of the system under test (SUT). Many MBT methods rely on building an automaton from the model and then they generate test cases by covering the automaton with different path coverage criteria. However, if a model of the SUT is a logical formula over some complex mathematical theories (such as set theory) it may be more natural or intuitive to apply coverage criteria directly over the formula. On the other hand, domain partition, i.e. the partition of the input domain of model operations, is one of the main techniques in MBT. Partitioning is conducted by applying different rules or heuristics. Engineers may find it difficult to decide what, where and how these rules should be applied. In this paper we propose a set of coverage criteria based on domain partition for set-based specifications. We call them testing strategies. Testing strategies play a similar role to path- or data-based coverage criteria in structural testing. Furthermore, we show a partial order of testing strategies as is done in structural testing. We also describe an implementation of testing strategies for the Test Template Framework, which is a MBT method for the Z notation; and a scripting language that allows users to implement testing strategies.