Exploring Richer and More Accurate Information via Frequency Selection for Image Restoration

Image restoration aims to recover high-quality images from their corrupted counterparts. Many existing methods focus on the spatial domain while overlooking frequency variations between sharp/degraded image pairs. Meanwhile, they typically establish skip connections between encoder and decoder features using addition or concatenation to enhance image restoration. However, since encoder features may contain degradation factors, this approach can inadvertently introduce implicit noise. In this paper, we introduce a multi-scale frequency selection network (MFSNet) that seamlessly integrates spatial and frequency domain knowledge, selectively recovering richer and more accurate information. Specifically, we initially capture spatial features and input them into dynamic filter selection modules (DFS) at different scales to integrate frequency knowledge. DFS utilizes learnable filters to generate high and low-frequency information and a frequency cross-attention mechanism (FCAM) to determine the most information to recover. To learn a multi-scale and accurate set of hybrid features, we develop a skip feature fusion block (SFF) that leverages contextual features to discriminatively determine which information should be propagated in skip-connections. It is worth noting that our DFS and SFF are generic plug-in modules that can be directly employed in existing networks without any adjustments, leading to performance improvements. Extensive experiments across various image restoration tasks demonstrate that our MFSNet achieves performance that is either superior or comparable to state-of-the-art algorithms. The code and the pre-trained models are released at https://github.com/Tombs98/MFSNet_.

I. Introduction

Image restoration is a longstanding low-level vision problem that seeks to recover a latent sharp image from its corrupted counterpart. It is an commonly ill-posed inverse problem, many conventional approaches [1], [2], [3], [4] address this by explicitly incorporating various priors or hand-crafted features to constrain the solution space to natural images. Nonetheless, designing such priors proves challenging and lacks generalizability.