Joint Intensity and Event Framework for Vehicle Detection in Degraded Conditions

Images captured by traditional cameras inevitably suffer from quality degradation under adverse conditions (e.g., low-light, snowy and foggy), this will jeopardize the safety of the self-driving system. Unlike traditional cameras, event cameras can work stably in high dynamic range scenarios, capture accurate motion clues of objects in motion scenes. In this paper, we propose a RGB-Event fusion framework to achieve robust vehicle detection in degraded conditions. Firstly, we design a dual-stream feature extraction network to extract modality-specific features. Secondly, we propose a Feature Align Module (FAM) to align the feature space of the two modalities. Finally, we present a simulated RGB-Event dataset for vehicle detection. It contains variable weather and lighting conditions collected in Panosim. Experimental results on the proposed dataset show that our proposed method can outperform the SOTA framed-based method and RGB-Event method by 2.4%, 7.7% in mAP50 and 4.2%, 8.9% in mAP50:95, respectively.