Image Super-Resolution via Generative Adversarial Network Using an Orthogonal Projection

In this paper, we propose a simple but powerful idea to improve super-resolution (SR) methods based on convolutional neural networks (CNNs). We consider a linear manifold, which is the set of all SR images whose downsampling results are the same as the input image, and apply the orthogonal projection onto this linear manifold in the output layers of the CNNs. The proposed method can guarantee the consistency between the SR image and the input image and reduce the mean squared error. The proposed method is especially effective for SR methods based on generative adversarial networks (GANs), composed of one generator and one discriminator, since the generator can learn high-frequency components while maintaining low-frequency ones. Experiments show the effectiveness of the proposed technique for a GAN-based SR method. Finally we introduce an idea of extension to noisy images.

I. Introduction

Super-resolution (SR) is a reconstruction problem of high-resolution (HR) images including various high-frequency components from low-resolution (LR) images including only low-frequency components [1]–[13]. In SR, it is important not only to increase the number of pixels but also to recover the original high-frequency components. In this paper, we focus on single image SR, which is an under-determined inverse problem since we have to recover a HR image from a single LR image having a smaller number of pixels. The simplest ways to increase the number of pixels are algebraic interpolations, e.g., the nearest-neighbor, bilinear, and bicubic interpolations. Although these algebraic methods are very fast, they cannot recover the high-frequency components at all. Therefore, SR results, called SR images in this paper, of the algebraic methods are very blurred.