A Benchmark for LiDAR-based Panoptic Segmentation based on KITTI

Panoptic segmentation is the recently introduced task that tackles semantic segmentation and instance segmentation jointly [18]. In this paper, we present an extension of SemanticKITTI [1], a large-scale dataset providing dense point-wise semantic labels for all sequences of the KITTI Odometry Benchmark [10]. This extension enables training and evaluation of LiDAR-based panoptic segmentation. We provide the data and discuss the processing steps needed to enrich a given semantic annotation with temporally consistent instance information, i.e., instance information that supplements the semantic labels and identifies the same instance over sequences of LiDAR point clouds. Additionally, we present two strong baselines that combine state-of-the-art LiDAR-based semantic segmentation approaches with a state-of-the-art detector enriching the segmentation with instance information and that allow other researchers to compare their approaches against. We believe that our extension of SemanticKITTI with strong baselines enables the creation of novel algorithms for LiDAR-based panoptic segmentation as much as it has for the original semantic segmentation and semantic scene completion tasks. Data, code, and an online evaluation service using a hidden test set are publicly available at http://semantic-kitti.org.