Managing Ambiguity in Regression Ensembles

We propose a regression ensemble based on a decomposition that separates the weighted average errors of individual learners and the ambiguity of their estimates. This approach is a modification of Gradient Boosting with a variation of the gradient at each step. That allows ensuring explicitly a diversity of base estimators. In addition, the proposed approach allows visualizing the contribution of individual estimators to the total ensemble error and evaluating the fitting to the training and validation data.