Densely Connected Transformer With Linear Self-Attention for Lightweight Image Super-Resolution

Image super-resolution (SR) is the process of restoring high-resolution (HR) images from low-resolution (LR) ones. Recent Transformer-based SR methods have achieved impressive results by utilizing the self-attention (SA) mechanism, which allows modeling long-range dependencies among input features in spatial dimensions. However, the computational complexity of SA increases quadratically with respect to the feature size, which makes Transformer-based methods inefficient. Additionally, despite the success of dense connections in convolutional neural network (CNN)-based methods, they have not been fully explored in Transformer-based methods. In this article, we propose a novel approach for lightweight SR, called densely connected transformer with linear SA (DCTLSA) network. Our method addresses the efficiency issue of SA by designing a new linear SA (LSA), which calculates the similarities in spatial dimension with linear complexity. Moreover, we leverage dense connections to integrate multiple levels of features and provide rich information for SR. Our experimental results demonstrate that DCTLSA outperforms state-of-the-art lightweight SR methods in terms of SR performance, model complexity, and inference speed. The code of the proposed method is available at https://github.com/zengkun301/DCTLSA.

I. Introduction

Images play a crucial role in acquiring valuable information for humans and have extensive applications in diverse fields including healthcare, remote sensing, and surveillance. However, various challenges arise in the context of image acquisition and analysis, primarily stemming from limitations in instrumentation or measurement techniques. Negative factors, such as the capabilities of digital image capture devices [1] and the influence of adverse environmental conditions [2], can significantly impact the quality of the acquired images. These challenges necessitate the development of advanced techniques and methodologies to overcome the limitations posed by the instrumentation and measurement process. One prominent challenge arising from these factors is the production of low-resolution (LR) images that lack essential details, rendering them unsuitable for direct use. To address this issue, image super-resolution (SR) [3], [4], aiming to reconstruct high-resolution (HR) images from their LR counterparts without the need for costly hardware upgrading, has been demonstrated usefully in an extensive range of fields, such as medical imaging [5], [6], remote sensing [7], and object detection [8].