Predicting software defects: A cost-sensitive approach

Find software defects is a complex and slow task which consumes most of the development budgets. In order to try reducing the cost of test activities, many researches have used machine learning to predict whether a module is defect-prone or not. Defect detection is a cost-sensitive task whereby a misclassification is more costly than a correct classification. Yet, most of the researches do not consider classification costs in the prediction models. This paper introduces an empirical method based in a COCOMO (COnstructive COst MOdel) that aims to assess the cost of each classifier decision. This method creates a cost matrix that is used in conjunction with a threshold-moving approach in a ROC (Receiver Operating Characteristic) curve to select the best operating point regarding cost. Public data sets from NASA (National Aeronautics and Space Administration) IV&V (Independent Verification & Validation) Facility Metrics Data Program (MDP) are used to train the classifiers and to provide some development effort information. The experiments are carried out through a methodology that complies with validation and reproducibility requirements. The experimental results have shown that the proposed method is efficient and allows the interpretation of the classifier performance in terms of tangible cost values.

I. Introduction

Software testing is an activity that still depends intensively on the human knowledge and effort. The testing tasks consume most of the available resources of a project. Therefore, it is very important to propose and investigate methods for automate software test activities, thereby making these tasks faster and cheaper. This paper presents prediction models that can aid in such an endeavor, but the main purpose is to present a new cost-sensitive way to assess them.