DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

Semantic segmentation of nighttime images plays an equally important role as that of daytime images in autonomous driving, but the former is much more challenging due to poor illuminations and arduous human annotations. In this paper, we propose a novel domain adaptation network (DANNet) for nighttime semantic segmentation without using labeled nighttime image data. It employs an adversarial training with a labeled daytime dataset and an unlabeled dataset that contains coarsely aligned day-night image pairs. Specifically, for the unlabeled day-night image pairs, we use the pixel-level predictions of static object categories on a daytime image as a pseudo supervision to segment its counterpart nighttime image. We further design a re-weighting strategy to handle the inaccuracy caused by misalignment between day-night image pairs and wrong predictions of daytime images, as well as boost the prediction accuracy of small objects. The proposed DANNet is the first one-stage adaptation framework for nighttime semantic segmentation, which does not train additional day-night image transfer models as a separate pre-processing stage. Extensive experiments on Dark Zurich and Nighttime Driving datasets show that our method achieves state-of-the-art performance for nighttime semantic segmentation.

1. Introduction

Aiming to label each pixel of a given image to an object category, semantic segmentation is a fundamental computer vision task and plays an important role in many applications such as autonomous driving [11], medical imaging [5] and human parsing [49]. With the advancement of deep learning and computing power, the state-of-the-art performance of semantic segmentation for natural scene images taken at the daytime has been significantly improved in recent years [10], [17]. Many researchers have started to segment more challenging images under various kinds of degradations, such as those taken in foggy weather [34] or at the nighttime [33]. In this paper, we focus on semantic segmentation of nighttime images, which has wide and important applications in autonomous driving.