Adaptive Fuzzy Degradation Perception Based on CLIP Prior for All-in-One Image Restoration

Despite substantial progress, the existing all-in-one image restoration methods still lack the ability to adaptively sense and accurately represent degradation information, thus hindering the enhancement of restoration performance. In addition, due to the large uncertainty and fuzziness of the data distribution in real scenarios compared to the training data, the model's generalization ability is often limited. To address the above issues, we propose a novel adaptive fuzzy degradation perception approach based on fuzzy theory that includes two tactics: 1) Fuzzy Degradation Perceiver (FDP); and 2) Test-time Self-supervised Prompt Fine-tuning (TSPF). On the one hand, we introduce the FDP, which leverages the rich visual language prior knowledge in CLIP to learn the prompt representations of different degradations. These prompts are regarded as semantic representations of various degradation fuzzy sets, achieving adaptive degradation perception by computing the degrees of membership between input images and the fuzzy sets. On the other hand, we propose the TSPF strategy, which is capable of self-supervised optimization of degraded fuzzy sets according to real-world scenarios during testing. This strategy improves the model's ability to perceive and represent the degraded information in data with real-world distributions. Thanks to the above key strategies, our method significantly improves degradation perception capability and image restoration quality while exhibiting excellent generalization in complex real-world scenarios. Extensive experiments on multiple benchmark datasets confirm that our approach achieves state-of-the-art performance in all-in-one image restoration.

I. Introduction

Image restoration (IR) endeavors to recover high-quality images from corrupted counterparts, playing a pivotal role in enhancing human perception and facilitating subsequent tasks like classification [1] and segmentation [2]. With the advancement of deep learning, numerous IR methods based on CNNs [3], [4], [5] and Transforms [6], [7] have been developed for various tasks, such as denoising [6], [8], [8], deraining [5], [7], and dehazing [9], [10]. While these methods have achieved commendable results in their specific tasks, they require different pretrained weights to handle different degradation types, resulting in higher storage costs and limited flexibility.