On Weight Design of Maximum Weighted Likelihood and an Extended EM Algorithm

The recent Maximum Weighted Likelihood (MWL) [18], [19] has provided a general learning paradigm for density-mixture model selection and learning, in which weight design, however, is a key issue. This paper will therefore explore such a design, and through which a heuristic extended Expectation-Maximization (X-EM) algorithm is presented accordingly. Unlike the EM algorithm [1], the X-EM algorithm is able to perform model selection by fading the redundant components out from a density mixture, meanwhile estimating the model parameters appropriately. The numerical simulations demonstrate the efficacy of our algorithm.

1 Introduction

As a general solution for the parameter estimate in a density mixture model, the Expectation-Maximization (EM) algorithm [1] and its variants, e.g., see [2], [3], [4], [5], [6] have been extensively applied to a variety of applications such as data clustering [7], Bayesian network [8], hidden Markov model [9], bioinformatics [10], and so forth. Nevertheless, the EM is unable to make a model selection, i.e., to determine the appropriate number of model components in a density mixture. If the number of components is not correctly assigned, the EM will generally lead to a poor estimate of the model parameters. In general, it is a nontrivial task to determine such a number.