High-Dimensional Constrained Huber Regression

In big data analytics, the collected data may be contaminated by heavy-tailed noises or outliers, and the sample size may be insufficient. In this paper, we study robust sparse regression under the presence of asymmetric heavy-tailed errors within a high-dimensional setting, where the ambient dimension can exceed the sample size. The estimation problem is formulated as an
$\ell_{1}$
constrained regression with Huber loss function. We propose a simple projected gradient descent algorithm to solve the problem and establish its convergence properties, accounting for both computational and statistical errors. Under mild conditions, we demonstrate that the successive iterates converge at a linear rate to an estimate within the statistical precision of the model. Numerical experiments validate the robust estimation performance of the proposed method across various heavy-tail distribution settings.