Research on Surface Environment Perception via Camera-LiDAR Sensor Fusion

Situational awareness is a key link for autonomous surface vessels (ASV) to achieve automatic navigation, and it is also the basis for ASV to accomplish obstacle avoidance and path planning. In order to provide multidimensional information features of dynamic and static obstacles on the water surface, this paper proposes a significant sensor fusion framework based on complementary camera and LiDAR information. The latest detector YOLOv8 is used to detect objects in the image. The LiDAR point clouds reflected from the target are clustered and the front and back frame clusters tracking is completed to obtain the position and motion characters of the target. The association between the 2D detections and the 3D trackers are devised as a data assignment task, which can find the best matches between camera and LiDAR observation results. The comprehensive experiments on real lake scenario have exhibited the effectiveness of our proposed framework.