Weakly Supervised Semantic Point Cloud Segmentation: Towards 10× Fewer Labels

Point cloud analysis has received much attention recently; and segmentation is one of the most important tasks. The success of existing approaches is attributed to deep network design and large amount of labelled training data, where the latter is assumed to be always available. However, obtaining 3d point cloud segmentation labels is often very costly in practice. In this work, we propose a weakly supervised point cloud segmentation approach which requires only a tiny fraction of points to be labelled in the training stage. This is made possible by learning gradient approximation and exploitation of additional spatial and color smoothness constraints. Experiments are done on three public datasets with different degrees of weak supervision. In particular, our proposed method can produce results that are close to and sometimes even better than its fully supervised counterpart with 10X fewer labels.

1. Introduction

Recent developments in point cloud data research have witnessed the emergence of many supervised approaches [19], [20], [33], [12], [29]. Most efforts of current research are dedicated into two tasks: point cloud shape classification (a.k.a. shape recognition) and point cloud segmentation (a.k.a. semantic segmentation). For both tasks, the success of the state-of-the-art methods is attributed mostly to the deep learning architecture [19] and the availability of large amount of labelled 3d point cloud data [16], [1]. Although the community is still focused on pushing forward in the former direction, we believe the latter issue, i.e. data annotation, is an overlooked bottleneck. In particular, it is assumed that all points for the point cloud segmentation task are provided with ground-truth labels, which is often in the range of 1k to 10k points for a 3d shape [34], [16]. The order of magnitude increases drastically to millions of points for a real indoor scene [11]. As a result, very accurate labels for billions of points are needed in a dataset to train good segmentation models. Despite the developments of modern annotation toolkits [16], [1] to facilitate large-scale annotation, exhaustive labelling is still prohibitively expensive for ever growing new datasets. Figure 1:

Illustration of the weak supervision concept in this work. Our approach achieves segmentation with only a fraction of labelled points‥