Fractal Residual Network and Solutions for Real Super-Resolution

The degradation function in single image super-resolution (SISR) is usually bicubic with an integer scale factor. However, bicubic is not realistic and a scale factor is not always an integer number in the real world. We introduce some solutions that are appropriate for realistic SR. First, we propose down-upsampling module which allows general SR network to use GPU memory efficiently. With the module, we can stack more convolutional layers, resulting in a higher performance. We also adopt a new regularization loss, auto-encoder loss. That loss generalizes down-upsampling module. Furthermore, we propose fractal residual network (FRN) for SISR. We extend residual in residual structure by adding new residual shells and name that structure FRN because of the self-similarity like the fractal. We show that our proposed model outperforms state-of-the-art methods and demonstrate the effectiveness of our solutions by several experiments on NTIRE 2019 dataset.

1. Introduction

Single image super-resolution (SISR) is an image restoration problem to get a high-resolution (HR) image from its low-resolution (LR) image downsampled with some degradation function. SISR is one of the important low-level computer vision tasks because it can be applied in various fields such as medical image [15], [22], satellite image [24], and surveillance [28]. Recently, deep learning based methods boost performance in the computer vision fields [3], [11], [17]. Similarly, deep neural networks has provided significant improvements in SISR [2], [4], [8], [9], [12], [13], [18], [19], [26], [27].