Semantic Map Construction via Multi-sensor Fusion

Semantic SLAM can achieve the acquisition of environmental semantic information and increase the understanding of the environmental content. Therefore, it has received extensive attention from academia and industry. In the most existing work, the visual SLAM and deep learning algorithms are combined, and the point cloud fusion method is used to build maps. It is difficult to be directly applied to robot navigation due to problems of time misalignment of sensor data, mismatched detection ranges, and inconsistent accuracy. In this paper, a method of building a semantic map is proposed by combining lidar, camera, odometer. First, the semantic information and position information of the target object are obtained by the YOLO v3 algorithm through the camera, and the grid map is constructed by using the Gmapping algorithm. Second, the semantic information and position information are merged into the grid semantic map to form a semantic grid map. Finally, the effectiveness and feasibility of this method are verified by these experiments.