Generative adversarial networks using variational autoencoder discrimination for super-resolution

Training generative adversarial networks (GANs) relies on the game between the generator and the discriminator, so the improvement of the discriminator can promote the improvement of the generator. A variational autoencoder (VAE) is useful for classification because it can learn the probability distribution of an image and provide latent variables as output features. Therefore, we propose a new network structure using a variational autoencoder in the GAN discriminator for super-resolution (SR). This network uses the latent variables generated by the VAE to extract the probability distribution of a reconstructed image and an original high-resolution (HR) image so the latent variables are used as features for discrimination. In addition, we propose to train the whole GAN and the VAE network alternately to optimize the network parameters. We verify the proposed method on five GAN-based image super-resolution methods. Experimental results show that the proposed algorithm leads to state-of-the-art results when plugged into existing GAN based SR methods.

I. Introduction

Single image super-resolution reconstructs HR images from one or more low-resolution images [1], [2], which is a hot topic in computer vision. In recent years, neural networks have been widely used in single image super-resolution(SR) of their nonlinear feature representation ability.