Robust Gaussian Mixture Modeling: A $\mathcal{K}$-Divergence Based Approach

This paper addresses the problem of robust Gaussian mixture modeling in the presence of outliers. We commence by introducing a general expectation-maximization (EM)-like scheme, called
$\mathcal{K}$
-BM, for iterative numerical computation of the minimum
$\mathcal{K}$
-divergence estimator (M
$\mathcal{K}$
DE). This estimator leverages Parzen's non-parametric
$\mathcal{K}$
ernel density estimate to down-weight low density regions associated with outlying measurements. Akin to the conventional EM, the
$\mathcal{K}$
-BM involves successive
M
aximizations of lower
B
ounds on the objective function of the M
$\mathcal{K}$
DE. However, differently from EM, these bounds are not exclusively reliant on conditional expectations. The
$\mathcal{K}$
-BM algorithm is applied to robust parameter estimation of a finite-order multivariate Gaussian mixture model (GMM). We proceed by introducing a new robust variant of the Bayesian information criterion (BIC) that penalizes the M
$\mathcal{K}$
DE's objective function. The proposed criterion, called
$\mathcal{K}$
-BIC, is conveniently applied for robust GMM order selection. In the paper, we also establish a data-driven procedure for selection of the kernel's bandwidth parameter. This procedure operates by minimizing an empirical asymptotic approximation of the mean-integrated-squared-error (MISE) between the underlying density and the estimated GMM density. Lastly, the
$\mathcal{K}$
-BM, the
$\mathcal{K}$
-BIC, and the MISE based selection of the kernel's bandwidth are combined into a unified framework for joint order selection and parameter estimation of a GMM. The advantages of the
$\mathcal{K}$
-divergence based framework over other robust approaches are illustrated in simulation studies involving synthetic and real data.