WeaFU: Weather-Informed Image Blind Restoration via Multi-Weather Distribution Diffusion

The extraction of distribution from images with diverse weather conditions is crucial for enhancing the robustness of visual algorithms. When addressing image degradation caused by different weather, accurately perceiving the data distribution of weather-informed degradation becomes a fundamental challenge. However, given the highly stochastic nature, modelling weather distribution poses a formidable task. In this paper, we propose a novel multi-Weather distribution difFUsion blind restoration model, named WeaFU. Firstly, the model employs representation learning to map image distribution into a latent space. Subsequently, WeaFU utilizes a diffusion-based approach, with the assistance of Diffusion Distribution Generator (DDG), to perceive and extract corresponding weather distribution. This strategy ingeniously injects data distribution into the recovery process, significantly enhancing the robustness of the model in diverse weather scenarios. Finally, a Conditional Distribution-Aware Transformer (CDAT) is constructed to align the distribution information with pixels, thereby obtaining clear images. Extensive experiments on real and synthetic datasets demonstrate that WeaFU achieves superior performance.

I. Introduction

In various visual tasks, the impact of extreme weather information on images is significantly amplified. This is particularly evident in tasks such as object detection [1], [2], [3], [4], [5], image segmentation [6], [7], [8], [9], and facial recognition [10], [11]. Confronted with such ill-posed problems, image restoration for various weather often poses even more intricate challenges. Due to the inability to determine the weather type of the current image, restoration models usually incur higher costs when learning this unstable degradation form.