Robust Regression Analysis Based on the K-Divergence

This paper presents a new framework for robust regression analysis. Under this framework, the input-output relation of a system is inferred by minimizing a new robust loss which relates the outputs to a presumed parametric function of the inputs. The considered loss arises from a modified version of the recently developed $\mathcal{K}$-divergence (tailored here for regression analysis), whose empirical estimate employs Parzen’s non-parametric kernel density estimator to mitigate the effect of low-density contaminations, attributed to outliers. The use of Parzen’s non-parametric density estimator provides a model-free weighting mechanism to mitigate the effect of outlying measurements in both input and output data sets. The performance advantage of the considered approach, over other robust regression methods, is illustrated in a simulation study focusing on robust training of a shallow GELU neural network.