RainGAN: Unsupervised Raindrop Removal via Decomposition and Composition

Adherent raindrops on windshield or camera lens may distort and occlude vision, causing issues for downstream machine vision perception. Most of the existing raindrop removal methods focus on learning the mapping from a raindrop image to its clean content by the paired raindrop-clean images. However, the paired real-world data is difficult to collect in practice. This paper presents a novel framework for raindrop removal that eliminates the need for paired training samples. Based on the assumption that a raindrop image is a composition of a clean image and raindrop style, the proposed framework decomposes a raindrop image into a clean content image and a raindrop-style latent code. Inversely, it composes a clean content image and a raindrop style code to a raindrop image for data augmentation. The proposed framework introduces a domain-invariant residual block to facilitate the identity mapping for the clean portion of the raindrop image. Extensive experiments on real-world raindrop datasets show that our network can achieve superior performance in raindrop removal to other unpaired image-to-image translation methods, even with comparable performance with state-of-the-art methods that require paired data.

1. Introduction

Adverse weather such as rain poses a challenge for outdoor computer vision tasks. Adherent raindrops on windshield or camera lens usually distort and occlude a portion of scene, leading to degraded performance of downstream computer vision applications including self-driving cars and outdoor surveillance cameras. Therefore, it is essential to restore a clear scene first.