Image Super-Resolution With Cross-Scale Non-Local Attention and Exhaustive Self-Exemplars Mining

Deep convolution-based single image super-resolution (SISR) networks embrace the benefits of learning from large-scale external image resources for local recovery, yet most existing works have ignored the long-range feature-wise similarities in natural images. Some recent works have successfully leveraged this intrinsic feature correlation by exploring non-local attention modules. However, none of the current deep models have studied another inherent property of images: cross-scale feature correlation. In this paper, we propose the first Cross-Scale Non-Local (CS-NL) attention module with integration into a recurrent neural network. By combining the new CS-NL prior with local and in-scale non-local priors in a powerful recurrent fusion cell, we can find more cross-scale feature correlations within a single low-resolution (LR) image. The performance of SISR is significantly improved by exhaustively integrating all possible priors. Extensive experiments demonstrate the effectiveness of the proposed CS-NL module by setting new state-of-the-arts on multiple SISR benchmarks.

1. Introduction

Single image super resolution (SISR) aims at recovering a high-resolution (HR) image from its low-resolution (LR) counterpart. SISR has numerous applications in the areas of satellite imaging, medical imaging, surveillance monitoring and high-definition display and imaging etc [3], [32], [40], [41], [43]. The mapping between LR and HR image is not bijective, which yields more possibilities for a faithful and high-quality HR recovery. Due to this ill-posed nature, SISR remains challenging in the past decades.