Does Detecting more Defects Contribute to Better Estimation? An Empirical Investigation to the Capture-Recapture Method

Capture-recapture method is a promising technique to estimate post-inspection residual defects in the inspected software artifacts. Since decisions such as whether a re-inspection is required usually rely on the estimated number of undiscovered defects, accuracy of the capture-recapture estimates hence become one of the critical factors impacting the decisions. Among many attempts to improve the accuracy of the capture-recapture estimates, most believe (implicitly or explicitly) that an effective approach to improve the estimation accuracy is to increase inspectors' capability to detect more defects. However, few empirical evidence has been reported to support this claim. For most capture-recapture estimators, the correlation is not crystal clear between the number of detected defects and the estimation accuracy. Therefore, it is necessary to empirically evaluate this claim so as to support the rationale that inspectors improve their capability on detecting more defects in order to achieve a higher estimation accuracy. We carried out an empirical study to investigate the relationship between the estimation accuracy (represented by Relative Error) and the percentage of the number of unique defects to the total number of seeded defects (denoted by Detection Rate). Results indicate that the above claim can be safely accepted in most cases, which supports the attempts to increasing capture-recapture estimation accuracy by improving inspectors' detection capability. However, there still could be chances that the strategy to improve estimation accuracy may fail, which implies that more strategies should be explored to improvement the application of the capture-recapture method.