LIV-LAM: LiDAR and Visual Localization and Mapping

This paper presents a framework for Simultaneous Localization and Mapping (SLAM) by combining a novel method for object discovery and localization from a monocular camera with depth information provided by Light Detection and Ranging (LiDAR). One major challenge in vision is discovering unknown objects without prior training/supervision, in the wild, and on-the-fly. In our framework, no training samples are available prior to the deployment. We develop an efficient proposal-matching method to discover object temporal saliency, and then finetune these frequently matched object proposals according to tracking information. Detected features of the objects are used as landmark features, and are merged with the LiDAR data in the proposed LIV-LAM (LiDAR and Visual Localization and Mapping). Compared to most visual SLAM or LiDAR-based SLAM, the novelty of this method is the computationally-efficient object detection and localization for feature set-and-match, in order to increase the accuracy of the generated map. The results show that the presented method is superior in both accuracy and efficiency of the maps generated by LiDAR.