Obstacle detection on tunnel walls based on fusion of images and point clouds

This paper proposes a novel attention weight ratio object detection model that integrates image and point cloud features for detecting obstacles on the tunnel wall facade. In previous researches, only point clouds were typically used as inputs for 3D object detection. However, in practical operation, point clouds may be obscured by water mist sprayed by cleaning vehicles. Thus, this paper introduces camera image data as supplementary input to address this issue. Based on this, this paper proposes a fusion module that uses attention dynamic selection mechanism to complement the fusion of point clouds and images. Compared with the traditional single modal model PointPillar and the classic multimodal model MV3D, our proposed model not only demonstrates excellent detection accuracy on the standard test set, but also demonstrates its excellent robustness in challenging water mist environments, resulting in a significant improvement in precision. These experimental results fully validate the advantages of our proposed bimodal feature fusion model in terms of performance and robustness. Therefore, this model effectively overcomes the problem of inaccurate obstacle detection under water mist interference and demonstrates its potential application in complex environments.