Automated Integration Testing Using Logical Contracts

Modern computer software usually consists of a number of interacting components. Defects pertaining to component interactions may not be visible until the components are integrated. To exercise component interactions, we propose an integration testing approach based on the core concept of contracts (i.e., preconditions and postconditions of component calls in first-order logic). To enable correctness analysis and generation of coverage-based tests (including state coverage, transition or component coverage, and reachability coverage) of a contract-based test model, we transform the test model into an operational model. We have implemented this approach into a framework that has the capacity to generate executable test code in a large collection of programming languages and to run the generated test code in a rich variety of test execution environments. To demonstrate the cost-effectiveness of our approach, we have conducted a series of experiments in controlled environments to measure the performance of test generation and the fault detection capabilities of coverage-based tests. The results have clearly demonstrated that: 1) our approach can quickly generate coverage-based tests when there is enough memory for dealing with the state space of the given test models; 2) reachability coverage is more capable of detecting faults than state coverage and transition coverage; 3) postcondition-based test oracles are more capable of detecting faults than state-based test oracles; and 4) robustness tests, which exercise the situations when the contracts are broken, are important for improving fault detection capability.