Residual Domain Super-Resolution Generative Adversarial Networks

Generative Adversarial Networks (GANs) deliver significant perceptual quality improvements over super-resolution methods using simple reconstruction losses. By combining adversarial training with perceptual loss functions, they avoid the over-smoothing resulting from the convergence towards the average of all plausible solutions. However, the likelihood of mode collapse if generator and discriminator learning rates are not carefully tuned or the training data does not represent the entire solution space, often introduces unpleasant artifacts that can compromise the overall quality of the upscaled images. We propose a new GAN architecture trained on the statistical distribution of a residue resulting from subtracting a spatial interpolation of the low-resolution (LR) image from the high-resolution (HR) ground-truth, instead of directly learning the pixel distribution of the HR image conditioned to the LR input. Since the statistical distribution of this residue is easier to model, the probability of mode collapse and consequent introduction of artifacts are mitigated, resulting in an algorithm competitive with much more computationally complex diffusion or transformer models.

I. Introduction

Super-resolution (SR) [1] refers to the process of generating high-resolution (HR) images or videos from low-resolution (LR) inputs and has been an important topic of research for several decades. Early methods relied on spatial interpolation techniques [2], but their inability to generate high-frequency details soon motivated the introduction of more complex approaches, such as statistical [3], prediction-based [4], patch-based [5] or edge-based methods [6]. The most significant advances were however delivered by emerging deep-learning (DL) techniques [7] and in particular by convolutional neural networks (CNNs).