An Empirical Analysis of Generative Adversarial Network Training Times with Varying Batch Sizes

Increasing the performance of a Generative Adversarial Network (GAN) requires experimentation in choosing the suitable training hyper-parameters of learning rate and batch size. There is no consensus on learning rates or batch sizes in GANs, which makes it a "trial-and-error" process to get acceptable output. Researchers have differing views regarding the effect of batch sizes on run time. This paper investigates the impact of these training parameters of GANs with respect to actual elapsed training time. In our initial experiments, we study the effects of batch sizes, learning rates, loss function, and optimization algorithm on training using the MNIST dataset over 30,000 epochs. The simplicity of the MNIST dataset allows for a starting point in initial studies to understand if the parameter changes have any significant impact on the training times. The goal is to analyze and understand the results of varying loss functions, batch sizes, optimizer algorithms, and learning rates on GANs and address the key issue of batch size and learning rate selection.

I. Introduction

Generative Adversarial Networks (GANs) were introduced by Ian Goodfellow [1] in 2014 and operate as shown in Fig. 1 [2]. GANs use a Generator Network (G) and a Discriminator Network (D) as seen in Fig. 1 to produce new samples of previously unseen data. G is trained to produce samples from an input noise vector and the result of the process is presented to D. A singular value is produced by D which the circuit attempts to determine whether the input data is from a real set of data or from the data produced by G. The output of D is then used as feedback to train G further to attempt to fool D into thinking that the synthetic input from G is instead from a real dataset.