Monocular Vision based Perception System for Nighttime Driving

The perception of objects around the vehicle is important for both advanced driving assistance system (ADAS) and autonomous driving systems. However, it is a huge challenge to recognize objects in a low-light environment. In this paper, we propose a monocular vision based perception system for nighttime driving. First, the transnational nighttime driving videos are collected, which are further processed into a low-light enhancement dataset and a nighttime object detection dataset for deep learning. Then, the GAN-based EnlightenGAN is trained for enhancing the low-light image. The CNN-based YOLOX is trained and inferred to detect objects. Next, to obtain reliable pixel points on objects, dense feature points are output on the enhanced image by LoFTR, which extracts global features through the transformer. On the other hand, the deep learning-based monodepth2 is used to generate a depth map from the enhanced image. Finally, the relative distance of the object is obtained by projecting and filtering the feature points on the depth map. The experiments show that this system can robustly detect vehicles and pedestrians on the road with distance in the nighttime driving dataset, which can provide effective information for vehicles and robots in nighttime scenes.