Minimum Error Entropy Estimation Under Contaminated Gaussian Noise

It is shown that Rényi's entropy of a Gaussian mixture with entropic index
$\alpha \in (1,\infty ]$
is upper-bounded by the cluster with minimum variance. This basic idea leads to a clean worst-case formulation of the minimum error entropy principle in the context of linear multi-sensor fusion by using a largely contaminated Gaussian distribution to model sensor errors with outliers. The obtained entropic best linear unbiased estimator leads to an operational interpretation in terms of a precision/reliability trade-off, it resonates closely with model-order selection methods, and it provides a possible information-theoretic root to sparsity-promoting regularization.