Functional Dependency Detection for Integration Test Cases

This paper presents a natural language processing (NLP) based approach that, given software requirements specification, allows the functional dependency detection between integration test cases. We analyze a set of internal signals to the implemented modules for detecting dependencies between requirements and thereby identifying dependencies between test cases such that: module 2 depends on module 1 if an output internal signal from module 1 enters as an input internal signal to the module 2. Consequently, all requirements (and thereby test cases) for module 2 are dependent on all the designed requirements (and test cases) for module 1. The dependency information between requirements (and thus corresponding test cases) can be utilized for test case prioritization and scheduling. We have implemented our approach as a tool and the feasibility is evaluated through an industrial use case in the railway domain at Bombardier Transportation (BT), Sweden.

1. Introduction

One of the main goals of software testing is detecting as many critical bugs in the system under test as possible, thus improving software quality. Having an efficient testing process may lead to an earlier fault detection [1] and thereby achieving better quality of the software product earlier. The concept of efficiency in a typical testing process can be interpreted in different ways, where all steps of the software testing life cycle (STLC) can be considered as candidates for efficiency improvement [2], [3]. One particular research direction that concerns test case selection, prioritization and scheduling has been researched to a greater extent as a way to improve testing efficiency. Previously, we formulated the problem of test case scheduling and prioritization as a multi criteria decision making problem [4]–[6]. We showed that in our specific context of vehicular integration level testing, multiple criteria affect test prioritization such as de- pendencies between test cases (and requirements), test cases execution time and requirement coverage. However, without automatically or semi-automatically inferring such criteria, the multi criteria decision making solution can be expensive and labour-intensive. More so, new test cases might be generated continuously (e.g., as part of a continuous integration environment), thus demanding the multi criteria decision making solution to adapt quickly to support timely decision making. One of the most influential criterion in our context is the existing dependencies between test cases. Identifying such dependencies requires a systematic approach that has to consider artifacts ranging from system architecture to testing data. In addition, a small change in the system under test may lead to generation of new test cases and therefore the dependency between test cases continuously changes. Unfortunately, there does not exist many explicit ways to capture such dependencies automatically as this problem is shown to be challenging as well as time and resource consuming [7]. However, identification of dependencies in test cases, especially at an early stage, promises to improve testing efficiency since dependent test cases are obvious candidates to be scheduled in a way that effectively test integration between implemented functions. In [8], we have shown through simulation that identifying the dependencies between test cases in the early stage of a testing process can help save test execution cost. Given the importance of identifying dependencies between test cases, this study answers the following research goal: