Sharpening Mixture of Experts Fusion of Infrared and Visible Images for Night Perception Enhancement

Most object detection frameworks for the autonomous vehicle only focus on the perception task in good illumination conditions, ignoring the task at night time. This paper aims to develop an accurate and real-time object detection framework using the combined information of the visible and infrared cameras. A dual-modal dataset is firstly established using our designed vision acquisition platform, the image pairs are all captured in the local traffic environment. Then the sharpening mixture of experts fusion model based on the state-of-the-art network YOLOv5 is devised to adaptively learn the complementary information of the visible and infrared modalities. Finally, comprehensive comparative experiments are implemented, the results show that our proposed fusion framework outperforms the other fusion methods at night perception scenarios.