2PS: A novel swarm based selection algorithm for ensemble learning problem

Discovering the effective subset of models in a pool of classifiers is an important and remarkable topic in ensemble learning scope. Using meticulously selected subset instead of entire ensemble leads to more efficient and effective results. This paper introduces a novel hybrid ensemble selection method of firefly and forward search algorithms. Because of the two different selection phases in the proposed method, it is called 2PS (Two-Phase Selection) method. Empirical comparisons of the method 2PS and two similar methods are performed on ten standard machine learning problems. The results show that the method 2PS leads to 5.63% average accuracy improvement compared to rivals. This great success is due to diversity balancing and then error correcting capability of 2PS, which is due to the nature of its firefly algorithm. Moreover, 2PS achieves second great success in overhead reduction by excluding redundant and weaker models in prediction which is due to its forward search algorithm.

I. Introduction

Ensemble method [1] is an important learning algorithm, which allows the possibility of combining the results of multiple learning algorithms. Ensemble method leads classifiers to benefit from their superior characteristics and to cover each other's errors [1]–[3]. However, ensemble method faces with some important challenges, e.g. high computation and commumcation overheads, low predictive performance, and low diversity [1]–[4]. To solve these challenges, recent researches [5]–[16] were proposed to add an intermediate phase for ensemble method, which is called ensemble selection or ensemble pruning phase. It is an especial algorithm which selects the more effective subset among initial ensemble.