Comparative Analysis of Ensembles Created Using Diversity Measures of Regressors

The paper presents a method for creating homogeneous and heterogeneous ensembles using three measures of regressor diversity, namely chi-square statistic, correlation coefficient and disagreement. Regressors included in the ensemble were selected from a pool of 100, 200 and 300 models generated using algorithms included in the WEKA data mining software package, such as M5P, M5Rules, Random Tree, and REPTree. Using three different measures, the degree of diversity of each pair of regressors was determined, and then 40 models with the highest diversity value were selected to construct ensembles. The obtained regressor ensembles were compared in terms of accuracy of prediction with models created using standard machine learning techniques such as linear regression, k-nearest neighbors, bagging and random forest. For comparative experiments, a real-world data set of purchase and sale transactions of residential real estate taken from a cadastral system was used. The research showed the usefulness of the proposed method for building ensembles.