Improving Effectiveness of Spectrum-based Software Fault Localization using Mutation Testing

The performance of software fault localization (SFL) is significant to software reliability and the software debugging process. Test suite effectiveness is crucial in improving the performance of spectrum-based fault localization (SBFL). Although code coverage metric (percentage of code merely executed by test cases) is a useful factor to assess the test cases quality but it has a drawback that it does not tell how much code actually checked and therefore this metric alone is not sufficient to measure the quality of test suites. For this reason, in this paper, we make use of mutation testing for the selection of effective test suites to improve the performance of spectrum-based fault localization (SBFL). We use two classic SBFL techniques Ochiai and Jaccard in our experiments to show the impact of test suite effectiveness on the performance of SBFL. We have used an open source mutation testing tool PIT to evaluate and select test cases and the unit testing framework JUnit has been used for writing and executing the test suites. Our experimental results show that the proposed approach can significantly improve the suspiciousness ranking of faulty statements. The results demonstrate that our approach effectively reduces the number of statements examined by an average of approx 50%.

I. Introduction

The use of software is ubiquitous in present world. We interact with software on a daily basis in some or the other way. The increasing size and complexity of software demands the need of maintaining software quality. Maintaining software quality and reliability is a challenging task and has become an important area of research. A fault or bug in a program causes an error condition that may lead to system failure. A failure of the system is a state when it behaves incorrectly.