Neural Network based Explicit Mixture Models and Expectation-maximization based Learning

We propose two neural network based mixture models in this work. The proposed mixture models are explicit. The explicit models have analytical forms with the advantages of computing likelihood and efficiency of generating samples. Expectation-maximization based algorithms are developed for learning parameters of the proposed models. We provide sufficient conditions to realize the expectation-maximization based learning. The main requirements are invertibility of neural networks that are used as generators and Jacobian computation of functional form of the neural networks. The requirements are practically realized using a flow-based neural network. In our first mixture model, we use multiple flow-based neural networks as generators. Naturally the model is complex. A single latent variable is used as the common input to all the neural networks. The second mixture model uses a single flow-based neural network as a generator to reduce complexity. The single generator has a latent variable input that follows a Gaussian mixture distribution. The proposed models are verified via training with expectation-maximization based algorithms on practical datasets. We demonstrate efficiency of proposed mixture models through extensive experiments for generating samples and maximum likelihood based classification.