Snow Removal in Video: A New Dataset and A Novel Method

Snowfall is a common weather phenomenon that can severely affect computer vision tasks by obscuring objects and scenes. However, existing deep learning-based snow removal methods are designed for single images only. In this paper, we target a more complex task - video snow removal, which aims to restore the clear video from the snowy video. To facilitate this task, we propose the first high-quality video dataset, which simulates realistic physical characteristics of snow and haze using a rendering engine and augmentation techniques. We also develop a deep learning framework for video snow removal. Specifically, we propose a snow-query temporal aggregation module and a snow-aware contrastive learning loss function. The module aggregates features between video frames and removes snow effectively, while the loss function helps identify and eliminate snow features. We conduct extensive experiments and demonstrate that our proposed dataset is more realistic than previous datasets, and the models trained on it achieve better performance in real-world snowing images. Our proposed method outperforms state-of-the-art video and image-based methods on both synthetic and real snowy videos.

1. Introduction

Snowfall is a common weather phenomenon that can significantly affect visibility in photographs, interfering with and reducing the accuracy of computer vision tasks such as target detection [34], tracking [14], and autonomous driving [33]. Image and video restoration [39, 23, 48, 37, 46, 7, 5, 26, 40, 43, 42, 41, 45, 31] is a widely studied direction, but research on image and video snow removal is still limited. Removing snow is crucial for improving accuracy and robustness of computer vision tasks. Additionally, snow can obstruct objects, decreasing image and video quality. Snow removal is a challenging task that requires distinguishing snowflake regions from the background and restoring obscured scenes, complicated by the complex geometric shape and texture of snowflakes. Moreover, snowfall often comes with haze, which further obscures the images.