Robust Multi-target Regression for Correlated Data Corruption

Multi-target regression has recently drawn increasing attention in the machine learning community due to its capability of simultaneously predicting multiple continuous target variables based on a given set of input features. Jointly handling the inter-target correlations and input-output relationships is very challenging. That task becomes even more intricate in the presence of correlated data corruption. We observe that traditional robust methods can hardly deal with several emerging challenges, including 1) presence of correlated corruption among targets in the datasets, 2) difficulty in estimating the data corruption ratio, and 3) scalability to massive datasets. This paper proposes a novel approach that addresses all the above challenges by developing a distributed robust regression algorithm. Specifically, the algorithm optimizes regression coefficients of each target in parallel with a heuristically estimated corruption ratio and then consolidates the uncorrupted set in two strategies: global consensus and majority voting. Also, we prove that our algorithm benefits from strong guarantees in terms of convergence rates and coefficient recovery, which can be applied as a generic framework for robust regression problems with correlated corruption property. Extensive experiments on synthetic and real-world datasets demonstrate that our algorithm is superior to existing methods in both effectiveness and efficiency.