Image Synthesis using Ensemble of Loss Functions: A Comparative Analysis using GAN Architecture

In this paper authors tries to show success of using ensemble of loss function on GAN network model. The model was able to generate quality synthetic data on MNIST handwritten dataset. The performance of the simple vanilla GAN model is evaluated for binary entropy loss and least square loss combined and compared for both of these loss functions separately. Instead of using only one single activation function we have used multiple activation functions such as Sigmoid., Tanh and ReLU in the last layer of the discriminator. Minimum loss value is counted among the pairs of transformed output vectors that are changed using different activation functions. In this case study we have used MNIST handwritten dataset. At last observations are made that training a model using ensemble of L2 loss and binary cross entropy loss functions produced significantly better results compared to using only L2 and only binary cross entropy loss function.

I. Introduction

Ian J. Goodfellow and his companion in the year 2014 proposed Generative Adversarial Network (GAN) a new Deep learning based model and was capable of generating synthetic data[1]. The new model GAN is well versed in domain of image and speech synthesis and any tabular data generation. Widely most of the deep learning model that uses algorithms and mathematics can be categorized into Regression Model, Discriminative model and Generative model. GAN provides users the utility dataset and thus deals with scarcity of data to train a deep learning model. Huge number of problem specific sample set is necessary for every deep learning model and so this paper discusses and experimentally proves efficiency of GAN model in this regard.