Bio-inspired wrapper-based feature selection: does the choice of metric matter?

Applications of bio-inspired algorithms to feature selection offering promising performance can be frequently encountered in the literature. Serving as wrappers, their application essentially boils down to the selection of a classifier and an appropriate metric for evaluating model performance. Although the choice of classifier was investigated to a viable extent, this is, however, not the case for the choice of metric and its impact on performance. Nevertheless, the utilisation of a wide variety of metrics is clearly apparent, albeit to different proportions. This raises the question of whether some metrics might be a better choice than others for bio-inspired wrapper-based methods. This paper sheds some light on this matter by comparing different metrics from different perspectives, like correlation, general performance and subset sizes. Unexpected results were obtained, to say the least, apart from some minor exceptions. Generally, no single metric appears to have an edge over the others. Yet, further investigation is certainly warranted.