Residual Local Feature Network for Efficient Super-Resolution

Deep learning based approaches has achieved great performance in single image super-resolution (SISR). However, recent advances in efficient super-resolution focus on reducing the number of parameters and FLOPs, and they aggregate more powerful features by improving feature utilization through complex layer connection strategies. These structures may not be necessary to achieve higher running speed, which makes them difficult to be deployed to resource-constrained devices. In this work, we propose a novel Residual Local Feature Network (RLFN). The main idea is using three convolutional layers for residual local feature learning to simplify feature aggregation, which achieves a good trade-off between model performance and inference time. Moreover, we revisit the popular contrastive loss and observe that the selection of intermediate features of its feature extractor has great influence on the performance. Besides, we propose a novel multi-stage warm-start training strategy. In each stage, the pre-trained weights from previous stages are utilized to improve the model performance. Combined with the improved contrastive loss and training strategy, the proposed RLFN outperforms all the state-of-the-art efficient image SR models in terms of runtime while maintaining both PSNR and SSIM for SR. In addition, we won the first place in the runtime track of the NTIRE 2022 efficient super-resolution challenge. Code will be available at https://github.com/fyan111/RLFN.

1. Introduction

SISR aims to reconstructed a high-resolution image from a low-resolution image. It is a fundamental low-level vision task and has a wide range of applications [9], [13], [40]. Currently, deep learning based approaches [2], [5], [11], [12], [18], [28], [29], [31], [32], [34], [36], [38], [39], [48], [49] have achieved great success and continuously improved the quality of reconstructed images. However, most of these advanced works require considerable computation costs, which makes them difficult to be deployed on resource-constrained devices for real-world applications. Therefore, it is essential to improve the efficiency of SISR models and design lightweight models that can achieve good trade-offs between image quality and inference time.