Spatial Adaptive Fusion Consistency Contrastive Constraint: Weakly Supervised Building Facade Point Cloud Semantic Segmentation

Semantic segmentation of building facade point clouds has diverse applications. The development of semantic segmentation methods is inextricably linked to datasets. The available building facade datasets suffer from a lack of abundant semantic categories and data completeness. To compensate for these shortcomings, we propose a new building facade dataset characterized by various categories and relatively complete 3-D building facades. In addition, most existing methods focus on fully supervised learning, which relies on manually labeling large-scale point cloud data and results in high time and labor costs. In this article, we propose an effective weakly supervised building facade segmentation approach, called spatial adaptive fusion consistency contrastive constraint (SAF-C3), to solve the above problem. We first design a multirandom point cloud augmentor as an auxiliary supervision branch to enhance the learning ability of the original network branch. Then, we present a spatial adaptive fusion (SAF) module to extract discriminative features for building facade point clouds. Finally, we propose a spatial consistency contrastive constraint to explore the contrastive property in feature space and to ensure the predictive consistency among the augmentation and original branches. The proposed method achieves a significant performance improvement against the state-of-the-art methods on two building facade point cloud datasets through extensive experiments. In particular, the performance of SAF-C3 with 1% labels significantly surpasses the baseline network with 100% labels.

I. Introduction

Building facade point cloud semantic segmentation is of great importance due to its broad application in urban reconstruction [1], [2], [3], damage assessments [4], [5], construction robots [6], AR/VR [7], and so on. The rapid development of deep-based semantic segmentation approaches is inseparable from the demand for large-scale datasets [8], [9], [10]. Thus, there is a significant incentive to construct building facade datasets. Building facades have well-established design principles, and their structural and stylistic complexity is markedly different from those of the shapes in common 3-D datasets such as S3DIS [8] and Semantic3D [10]. This makes building facade segmentation a more unique challenge. Recently, Su et al. [11] proposed the first large-scale building facade point cloud dataset, XIAMEN-buildings, which covers approximately 3 km and over 160 million manually annotated points acquired by mobile laser scanners along city streets. Although it has various unique structural characteristics, rich semantic categories and complete 3-D building facades are absent. Incomplete building facades lose much crucial structural information, adversely affecting subsequent tasks (e.g., segmentation and reconstruction). To compensate for these deficiencies, we propose a new building facade dataset, termed PARIS-CARLA-buildings, which is extended from the existing Paris-CARLA-3-D dataset [12]. As shown in Fig. 1, the PARIS-CARLA-buildings dataset is characterized by relatively complete 3-D building facades as its principal part and has ten semantic categories. Fig. 1.

(a) and (b) (Top) Performance of SAF-C3 on the XIAMEN-buildings [11] and PARIS-CARLA-buildings datasets with two weakly supervised semantic segmentation settings: annotating 0.1% and 1%. The comparison methods include PSD [13], SQN [14], and baseline [15] with a 100% label. (Bottom) Ground-truth visualization of a sample from the corresponding dataset. (a) XIAMEN-buildings. (b) PARIS-CARLA-buildings.