FMambaIR: A Hybrid State-Space Model and Frequency Domain for Image Restoration

With the development of deep learning, impressive progress has been made in the field of image restoration. The existing methods mainly rely on CNNs and Transformers to obtain multiscale feature information. However, these methods rarely integrate frequency-domain information effectively during feature extraction, limiting their performance in image restoration. Additionally, few have combined Mamba with the Fourier domain for image restoration, which limits Mamba’s ability to perceive global degradation in the frequency domain. Therefore, we propose a new image restoration model called FMambaIR, which utilizes the complementarity between frequency and Mamba for image restoration. The core of FMambaIR is the F-Mamba block, which combines Fourier transform and Mamba for global degradation perception modeling. Specifically, F-Mamba adopts a dual-branch complementary structure, including spatial Mamba branches and Fourier frequency-domain global modeling. Mamba models the long-range dependencies of the entire image features, and the frequency branch utilizes Fourier to extract global degraded features from the image. Finally, we use a forward feedback network to integrate local information, which is beneficial for improving the recovery details. We comprehensively evaluate FMambaIR on several image restoration tasks, including underwater image enhancement, remote sensing image dehazing, and low-light image enhancement. The experimental results demonstrate that FMambaIR not only achieves superior performance compared to state-of-the-art methods but also significantly reduces computational complexity. Our code is available at https://github.com/mickoluan/FMambaIR.

I. Introduction

Lately, image restoration is a critical area in the field of computer vision, focusing on reconstructing or enhancing images that have degraded due to various factors, including noise, motion blur, and environmental conditions. Tasks such as remote sensing image dehazing [1], underwater image enhancement [2], and low-light image enhancement [3], [4] are encompassed in this domain. These tasks play a significant role in the advancement of higher-level computer vision tasks.