Empirical comparison of machine learning algorithms for bug prediction in open source software

Bug tracking and analysis truly remains one of the most active areas of software engineering research. Bug tracking results may be employed by the software practitioners of large software projects effectively. The cost of detecting and correcting the defect becomes exponentially higher as we go from requirement analysis to the maintenance phase, where defects might even lead to loss of lives. Software metrics in conjunction with defect data can serve as basis for developing predictive models. Open source projects which encompass contributions from millions of people provide capacious dataset for testing. There have been diverse machine learning techniques proposed in the literature for analyzing complex relationships and extracting useful information from problems using optimal resources and time. However, more extensive research comparing these techniques is needed to establish superiority of one technique over another. This study aims at comparison of 14 ML techniques for development of effective defect prediction models. The issues addressed are 1) Construction of automated tool in Java to collect OO, inheritance and other metrics and detect bugs in classes extracted from open source repository, 2) Use of relevant performance measures to evaluate performance of predictive models to detect bugs in classes, 3) Statistical tests to compare predictive capability of different machine learning techniques, 4) Validation of defect prediction models. The results of the study show that Single Layer Perceptron is the best technique amongst all the techniques used in this study for development of defect prediction models. The conclusions drawn from this study can be used for practical applications by software practitioners to determine best technique for defect prediction and consequently carry out effective allocation of resources.