DVMSR: Distillated Vision Mamba for Efficient Super-Resolution

Efficient Image Super-Resolution (SR) aims to accelerate SR network inference by minimizing computational complexity and network parameters while preserving performance. Existing state-of-the-art Efficient Image Super-Resolution methods are based on convolutional neural networks. Few attempts have been made with Mamba to harness its long-range modeling capability and efficient computational complexity, which have shown impressive performance on high-level vision tasks. In this paper, we propose DVMSR, a novel lightweight Image SR network that incorporates Vision Mamba and a distillation strategy. The network of DVMSR consists of three modules: feature extraction convolution, multiple stacked Residual State Space Blocks (RSSBs), and a reconstruction module. Specifically, the deep feature extraction module is composed of several residual state space blocks (RSSB), each of which has several Vision Mamba Moudles(ViMM) together with a residual connection. To achieve efficiency improvement while maintaining comparable performance, we employ a distillation strategy to the vision Mamba network for superior performance. Specifically, we leverage the rich representation knowledge of teacher network as additional supervision for the output of lightweight student networks. Extensive experiments have demonstrated that our proposed DVMSR can outperform state-of-the-art efficient SR methods in terms of model parameters while maintaining the performance of both PSNR and SSIM. The source code is available at https://github.com/nathan66666/DVMSR.git

1. Introduction

Single image super-resolution (SR) is a key challenge in computer vision and image processing, aiming to reconstruct a high-resolution image from a low-resolution input. Effective super-resolution aims to improve the efficiency of the SR model while maintaining reconstruction performance. Since the introduction of deep learning into super-resolution tasks [9], many CNN-based methods have been proposed [8], [10], [11], [25], [26], [29], [34] to improve the performance. A series of approaches [10], [17], [19], [25], [26], [28], [30], [36], [56] have been proposed for building efficient models for image SR. The majority of these efficient models focus on five factors: runtime, parameters, FLOPS, activations, and depths. To further promote the development of efficient SR, ICCV holds the first competition in the AIM 2019 challenge [60]. The information multi-distillation network(IMDN) [19] proposes cascaded information multi-distillation blocks to improve the feature extraction module, which won first place in this competition. After that, The winning solution of the AIM 2020 challenge [61], residual feature distillation network(RFDN) [36], further improves the IMDN by residual learning in the main block. In the efficient SR track of NTIRE 2022 [24] challenge, the winning solution, residual local feature network(RLFN) [28], removes the hierarchical distillation connection of residual feature distillation block(RFDB) [36] to reduce the inference time. In the efficient SR track of NTIRE 2022 [56] challenge, the winning solution utilizes a multi-stage lightweight training strategy that combines distillation and pruning to reduce both time consumption and model size.