Test Case Prioritization Based on Method Call Sequences

Test case prioritization is widely used in testing with the purpose of detecting faults as early as possible. Most existing techniques exploit coverage to prioritize test cases based on the hypothesis that a test case with higher coverage is more likely to catch bugs. Statement coverage and function coverage are the two widely used coverage granularity. The former typically achieves better test case prioritization in terms of fault detection capability, while the latter is more efficient because it incurs less overhead. In this paper we argue that static information such as statement and function coverage may not be the best criteria for guiding dynamic executions. Executions that cover the same set of statements /functions can may exhibit very different behavior. Therefore, the abstraction that reduces program behavior to statement/function coverage can be too simplistic to predicate fault detection capability. We propose a new approach that exploits function call sequences to prioritize test cases. This is based on the observation that the function call sequences rather than the set of executed functions is a better indicator of program behavior. Test cases that reveal unique function call sequences may have better chance to encounter faults. We choose function instead of statement sequences due to the consideration of efficiency. We have developed and implemented a new prioritization strategy AGC (Additional Greedy method Call sequence), that exploit function call sequences. We compare AGC against existing test case prioritization techniques on eight real-world open source Java projects. Our experiments show that our approach outperforms existing techniques on large programs (but not on small programs) in terms of bug detection capability. The performance shows a growth trend when the size of program increases.