SP-GAN: Self-Growing and Pruning Generative Adversarial Networks

This article presents a new Self-growing and Pruning Generative Adversarial Network (SP-GAN) for realistic image generation. In contrast to traditional GAN models, our SP-GAN is able to dynamically adjust the size and architecture of a network in the training stage by using the proposed self-growing and pruning mechanisms. To be more specific, we first train two seed networks as the generator and discriminator; each contains a small number of convolution kernels. Such small-scale networks are much easier and faster to train than large-capacity networks. Second, in the self-growing step, we replicate the convolution kernels of each seed network to augment the scale of the network, followed by fine-tuning the augmented/expanded network. More importantly, to prevent the excessive growth of each seed network in the self-growing stage, we propose a pruning strategy that reduces the redundancy of an augmented network, yielding the optimal scale of the network. Finally, we design a new adaptive loss function that is treated as a variable loss computational process for the training of the proposed SP-GAN model. By design, the hyperparameters of the loss function can dynamically adapt to different training stages. Experimental results obtained on a set of data sets demonstrate the merits of the proposed method, especially in terms of the stability and efficiency of network training. The source code of the proposed SP-GAN method is publicly available at https://github.com/Lambert-chen/SPGAN.git.

I. Introduction

A generative model is capable of synthesizing (or generating) a variety of new samples in a high-dimensional subspace [1]–[5]. Recently, generative adversarial network (GAN) [6], [7] has become one the most popular and successful generative models that are able to generate high-quality realistic images and videos. In contrast to some traditional deep neural network (DNN)-based generative models [8]–[11], such as the pixel recurrent neural networks (Pixel-RNNs) [12] and variational autoencoder (VAE) [13], GAN can generate high-quality virtual samples with low computational complexity and has no lower limit of variation. The various counterproductive generative networks are asymptotically consistent, while VAE has some bias [14]–[16]. Meanwhile, GAN has neither a lower limit of change nor a tricky partition function compared with deep Boltzmann machines (DBMs) [17]. Finally, virtual samples can be generated by GAN in a single forward pass, rather than through an iterative process, such as the Markov chain operators [18].