Removing Rain from Single Images via a Deep Detail Network

We propose a new deep network architecture for removing rain streaks from individual images based on the deep convolutional neural network (CNN). Inspired by the deep residual network (ResNet) that simplifies the learning process by changing the mapping form, we propose a deep detail network to directly reduce the mapping range from input to output, which makes the learning process easier. To further improve the de-rained result, we use a priori image domain knowledge by focusing on high frequency detail during training, which removes background interference and focuses the model on the structure of rain in images. This demonstrates that a deep architecture not only has benefits for high-level vision tasks but also can be used to solve low-level imaging problems. Though we train the network on synthetic data, we find that the learned network generalizes well to real-world test images. Experiments show that the proposed method significantly outperforms state-of-the-art methods on both synthetic and real-world images in terms of both qualitative and quantitative measures. We discuss applications of this structure to denoising and JPEG artifact reduction at the end of the paper.

1. Introduction

Under rainy conditions, the impact of rain streaks on images and video is often undesirable. In addition to a subjective degradation, the effects of rain can also severely affect the performance of outdoor vision systems, such as surveillance systems. Effective methods for removing rain streaks are needed for a wide range of practical applications. However, when an object's structure and orientation is similar with that of rain streaks, it is hard to simultaneously remove rain and preserve structure. To address this difficult problem, we develop an end-to-end deep network architecture for removing rain from individual images. Figure 1 shows an example of a real-world test image and our result. To date, many methods have been proposed for removing rain from images. These methods fall into two categories: video-based methods and single-image based methods. We briefly review these approaches and then discuss the contributions of our proposed framework. Figure 1:

An example real-world rainy image and our result.