Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion

Given the prominence of current 3D sensors, a fine-grained analysis on the basic point cloud data is worthy of further investigation. Particularly, real point cloud scenes can intuitively capture complex surroundings in the real world, but due to 3D data’s raw nature, it is very challenging for machine perception. In this work, we concentrate on the essential visual task, semantic segmentation, for large-scale point cloud data collected in reality. On the one hand, to reduce the ambiguity in nearby points, we augment their local context by fully utilizing both geometric and semantic features in a bilateral structure. On the other hand, we comprehensively interpret the distinctness of the points from multiple resolutions and represent the feature map following an adaptive fusion method at point-level for accurate semantic segmentation. Further, we provide specific ablation studies and intuitive visualizations to validate our key modules. By comparing with state-of-the-art networks on three different benchmarks, we demonstrate the effectiveness of our network.

1. Introduction

As 3D data acquisition techniques develop rapidly, different types of 3D scanners, e.g. LiDAR scanners [22] and RGB-D cameras [10] are becoming popular in our daily life. Basically, 3D scanners can capture data that enables AI-driven machines to better see and recognize the world. As a fundamental data representation, point clouds can be easily collected using 3D scanners, retaining abundant information for further investigation. Therefore, point cloud analysis is playing an essential role in 3D computer vision.