Image Synthesis using Ensemble of Loss Functions: A Comparative Analysis using GAN Architecture

In this paper authors tries to show success of using ensemble of loss function on GAN network model. The model was able to generate quality synthetic data on MNIST handwritten dataset. The performance of the simple vanilla GAN model is evaluated for binary entropy loss and least square loss combined and compared for both of these loss functions separately. Instead of using only one single activation function we have used multiple activation functions such as Sigmoid., Tanh and ReLU in the last layer of the discriminator. Minimum loss value is counted among the pairs of transformed output vectors that are changed using different activation functions. In this case study we have used MNIST handwritten dataset. At last observations are made that training a model using ensemble of L2 loss and binary cross entropy loss functions produced significantly better results compared to using only L2 and only binary cross entropy loss function.