MWRC3D: 3D Object Detection with Millimeter-Wave Radar and Camera Fusion

Sensor fusion has become an active research field due to its numerous advantages, such as improved perception capabilities, enhanced environment understanding, and better object detection and tracking performance. However, existing multi-sensors fusion models primarily concentrate on the combination of lidar and camera and neglect the millimeter-wave radar (MWR), an inexpensive and promising sensor. To address this limitation, this paper integrates camera and MWR, and proposes a novel framework called MWRC3D to achieve low cost and efficient 3d object detection. Specifically, we first propose an attention-based deep layer aggregation (ADLA) module for learning global associations and dependencies between individual pixels in an image, which improves the ability to characterize image features. Then, we introduce deformable convolutional networks (DCNs) to model geometric transformations, and a MWR data enhancement module is employed to correct 3D offset of radar point cloud from image center point. Finally, we stitch and fuse the image features with the radar feature maps as input to the quadratic regression head to obtain accurate 3D object detection boxes. To evaluate the effectiveness of the proposed model, extensive experiments are conducted based on the nuScenes dataset. The results demonstrate that the demonstrate that our method outperforms all baselines while maintaining high efficiency.