Multi-Domain Thermal Object Detection Using Generative Adversarial Networks

The major recent advances in autonomous driving are mostly based on recent progress in object detection methods. Object detection broadly depends on the quality of the cameras and sensors used to generate the images. Many kinds of sensors are being utilized for the sake of improving scene perception, such as RGB, radar, and LiDAR. However, the usage of all these sensors' data leads to object detection degradation in adverse lighting conditions. Thermal imaging depicts the spatial distribution of temperature differences in a scene. This leads to the ability of a thermal camera to outperform other sensors in severe weather conditions. In this research, we propose the use of thermal imaging to improve object detection in autonomous driving. The proposed framework inherits the architecture of EfficientDet and further proposes a technique to fuse the features of two different modalities: thermal and RGB. We use GAN as an I2I translation framework to transform the thermal images into RGB images. The experimental results demonstrate that the proposed framework outperforms the detection accuracy of other state-of-the-art object detectors in thermal imagery. We propose two variants, MDFFTDet-D0 and MDFFTDet-D2 which achieve mAP of 63.15% and 77.81% respectively on the FLIR ADAS dataset.