Detection criteria for evaluation of computer aided diagnosis systems

Many evaluation criteria can be used to measure the performance of a detection algorithm. Here, the authors examine the effect they have on the outcome. Two main choices are addressed: the size of the annotated region, and the criterion to decide if such a region is detected. The detection program the authors use automatically detects stellate lesions in digital mammograms. The authors show that both the size of the annotated region as well as the criterion for detection have an effect on the FROG curve. For fair comparison between systems, the chosen criteria should be the same, particularly at low specificity levels.