Improving Diversity in Concept Drift Ensembles

This research investigates how the performance of ensembles is influenced by the diversity and behavior of base learners on abrupt and gradual concept drifts. To accomplish that, we propose to enhance ensemble algorithms by adjusting the voting procedure and using heterogeneous base-learners on three benchmark data stream methods: Fast Adaptive Stacking of Ensembles (FASE), Online Adaptive Classifier Ensemble based on bagging (OACE
BA
), and Online Adaptive Classifier Ensemble based on boosting (OACE
BO
). These ensembles were selected because they use different voting schemes (stacking, simple majority, and weighted voting) to perform the classification. The modified methods, MFASE, MOACE
BA
and MOACE
BO
, were compared to their original versions regarding accuracy. Naïve Bayes (NB) and Hoeffding Tree (HT) were used as base learners to evaluate the methods on artificial and real-world datasets. The experimental investigation included 44 experiments and their evaluation using the Friedman and Nemenyi statistical tests. They showed that the modified methods improve the ensemble performance in concept drift problems in many scenarios.