More is not Always Better: Insights from a Massive Comparison of Meta-heuristic Algorithms over Real-Parameter Optimization Problems

Much controversy has been lately risen around the design and performance of modern bio-inspired optimization methods, in particular due to the alleged lack of algorithmic novelty in their definition with respect to traditional heuristic solvers. In this work we present a first attempt at shedding empirical evidence over this debate, for which results of a benchmark with unprecedented scales in terms of problems and algorithms are reported and discussed. Specifically, informed conclusions are held in what refers to the claimed superior performance of these bio-inspired solvers and their competitiveness when compared to competition-winning alternatives. Finally, we prove that the tailored selection of a subset of problems and techniques can unfairly bias the comparisons favoring any of such algorithms, ultimately arriving at illusory conclusions about their comparative performance.

I. Introduction

The last decade has witnessed the vast arrival of new meta-heuristic optimization algorithms, most relying on the emulation of behavioral patterns and processes observed in nature, not only in what refers to biological aspects but also to physical phenomena [1], [2]. Many studies have underscored the proliferation of contributions along this line in almost any venue related to Artificial Intelligence [3], [4]. In general, such contributions either present a new technique and evaluate it over synthetic and/or real-world optimization problems [5]. This literature spans even further with combinations (hybridizations) of algorithmic elements belonging to different solvers, yielding hybrid methods that encompass diverse search operators, archival strategies and other mixtures [6].