Prototype-Guided Multitask Adversarial Network for Cross-Domain LiDAR Point Clouds Semantic Segmentation

Unsupervised domain adaptation (UDA) segmentation aims to leverage labeled source data to make accurate predictions on unlabeled target data. The key is to make the segmentation network learn domain-invariant representations. In this work, we propose a prototype-guided multitask adversarial network (PMAN) to achieve this. First, we propose an intensity-aware segmentation network (IAS-Net) that leverages the private intensity information of target data to substantially facilitate feature learning of the target domain. Second, the category-level cross-domain feature alignment strategy is introduced to flee the side effects of global feature alignment. It employs the prototype (class centroid) and includes two essential operations: 1) build an auxiliary nonparametric classifier to evaluate the semantic alignment degree of each point based on the prediction consistency between the main and auxiliary classifiers and 2) introduce two class-conditional point-to-prototype learning objectives for better alignment. One is to explicitly perform category-level feature alignment in a progressive manner, and the other aims to shape the source feature representation to be discriminative. Extensive experiments reveal that our PMAN outperforms state-of-the-art results on two benchmark datasets.

I. Introduction

Semantic segmentation of 3-D LiDAR point clouds is of great significance to 3-D spatial perception and understanding, including smart cities [1], [2], 3-D map construction [3], autonomous driving [4], [5], etc. Although the performance of supervised segmentation methods [6], [7], [8], [9] has grown immensely, the improved performance is accompanied by laborious manual pointwise annotations. To relieve the heavy burden of human labor, we intend to label the real-world data with the help of synthetic data whose annotations can be obtained at almost zero cost. However, the model trained by synthetic data (source domain) cannot guarantee reliable performance on the real-world data (target domain) due to the different data distributions (domain shift). Thus, researchers have recently explored extensive works called transfer learning or unsupervised domain adaptation (UDA) to address this issue [10].