Density-Aware and Semantic-Guided Fusion for 3-D Object Detection Using LiDAR-Camera Sensors

Three-dimensional object detection plays a key role in autonomous driving systems. The performance of light detection and ranging (LiDAR)-based models is limited because of the sparsity and inhomogeneity of point clouds. Researchers have developed numerous camera-LiDAR-based fusion methods to address the limitations of LiDAR-based models. Still, because of the major differences in representations between pixel and point cloud, these fusion methods cannot appropriately leverage the benefits of both types of sensors. In this study, we developed a 3-D object detection model based on LiDAR-camera fusion. The model first augments raw points with possibility features extracted from a camera stream and subsequently uses possibility-guided farthest point sampling (P-FPS) for high-quality feature extraction in the 3-D backbone. In addition, we designed a detection head for density-aware artificial point sampling and fusion that can aggregate geometric and semantic features by using high-resolution artificial points and shift the position of an artificial point according to the density of points in a region of interest (ROI) to obtain more representative point features. We conducted massive experiments by using the public KITTI dataset. Our method outperforms previously developed state-of-the-art 3-D object detection methods, especially in detecting small and distant objects.