MCMC sampling on latent-variable space of mixture of probabilistic PCA

Cluster analysis is widely used in computational intelligent systems and pattern recognition fields. One of its main tasks is to detect unobservable labels showing to which clusters the given observable data belong. Recently, the accuracy of clustering has been formulated and asymptotically analyzed in a distribution-based manner. The results claim that the Bayes method is more accurate than the maximum-likelihood method. However, the computational cost of parameter marginalization is expensive in the Bayes method, which implies that the marginalization must be analytically calculated for practical use. The present paper focuses on a mixture of probabilistic principal component analysis, which is a constrained normal mixture model employed for advanced data analysis, and does not have analytic marginalization. We propose a sampling method for the label estimation based on a mixture of more general components, which have analytic marginalization. Experimental evaluation shows that the proposed method has advantages compared to the maximum-likelihood method, especially when the number of labels is unknown.

I Introduction

Mixture models are employed for cluster analysis in pattern recognition and intelligent data-processing systems. The models consist of various components; the structure allows us to apply it to a complicated data density, which cannot be dealt with by single-component models. The data have unobservable labels showing which component they belong to. The aim of cluster analysis based on this model is to detect these labels from the observable part of the data.