GLGFN: Global-Local Grafting Fusion Network for High-Resolution Image Deraining

Image deraining is a hot research topic, which aims to remove various rain streaks (raindrops) from rainy images and restore the backgrounds. Though image deraining has been extensively studied in recent years, few methods are able to effectively and efficiently derain real-world high-resolution rainy images. In general, existing image deraining methods are restricted by two main factors while processing high-resolution images. First, the computational complexity and memory usage of existing deep learning-based methods are high when it comes to derain high-resolution images. Second, as the image resolution increases, it is difficult to simultaneously extract and aggregate both global and local features for clean rain removal. In this paper, we propose a novel network, called Global-Local Grafting Fusion Network (GLGFN), for deraining real-world high-resolution images. Our GLGFN utilizes a staggered connection structure to achieve deeper sampling depth while maintaining low computational cost. It adopts the Transformer and CNN based encoders (backbones) to extract global and local features, respectively, and then grafts global features into local features to guide the extraction of rain streaks. In addition, for well fusing global and local features, we also propose a Grafting Fusion Module (GFM), which adopts Cross Sparse Attention (CSA) and Selective Kernel Fusion (SK Fusion) to efficiently aggregate global and local features. Extensive experiments conducted on several high-resolution real rainy datasets have demonstrated the effectiveness and efficiency of our proposed GLGFN. We will release our code and dataset.

I. Introduction

As a common weather phenomenon, raining, especially heavy raining conditions, may seriously degrade the quality of captured images and videos, and then affect the performance of subsequent computer vision tasks. For example, outdoor vision systems, such as autonomous driving, would be severely affected by rainy weather conditions. Thanks to the great advances of deep learning in recent years, significant progress has been achieved in single image deraining. However, it is still challenging to accurately detect rain streaks with various appearances and remove them from real-world rainy images, especially from high-resolution rainy images, though single image deraining is typically considered as a low-level vision problem.