PDAST: Paradigm Towards Weather Domain Adaptation for 3D Detection Based on Density Aware Pooling and Self-Training

Most 3D detection algorithms are developed and validated on datasets collected during clear weather, while performance significantly deteriorates under severe weather conditions such as snowy days. To address this problem, we propose PDAST, an unsupervised pipeline for weather domain adaptation. PDAST involves point density dealing (PDD), a data augmentation method, and a density-aware pooling module (PDV), both of which are plug-and-play modules to enhance algorithmic robustness in snowy weather conditions along with the self-training [1] strategy. The data augmentation method PDD randomly samples and perturbs the ground truth point database in the point cloud. Meanwhile, the PDV module combines voxel features and density-based positional encoding features as inputs to a multi-head self-attention module, facilitating the capture of global features comprising density features. In this paper, we validate the effectiveness of PDAST on a typical voxel-based 3D detection baseline, SecondIOU [2], and conduct experiments on STF [3] datasets with clear and snowy datasets. Our approach PDAST achieves a performance enhancement of around 14.9% ~ 25.5% for $AP_{3D}$ (Average Precision Of 3D Boundingboxes) and 17.5% ~ 29.3% for $AP_{AOS}$ (Average Precision Of Orientation Similarity) compared to the baseline detector on clear-to-snowy conditions. Furthermore, PDAST's performance in unsupervised clear-to-snowy conditions matches or exceeds the baseline model fully supervised on snowy conditions. Our PDAST alleviates the performance degradation caused by severe weather, such as snowy days, and is of great significance for all-weather perception.

I. Introduction

3D detection based on point cloud has achieved significant advancements in recent years, while the majority of the models are trained and validated on datasets that focus on a single domain characterized by the same or similar scenarios. When these algorithms are deployed in different domains, noticeable performance degradation occurs. Domain adaptation including transformation across sensors, regions, and weather conditions. Obtaining datasets covering all possible domains for training is impractical. Therefore, domain adaptation and performance generalization are of critical importance for 3D detection. Light Detection and Ranging (Lidar), as an active sensor, perceives the environment through the reflected point cloud from the target. Notable disparities exist in point quality, point density, and target size across different domains. For cross-weather domain adaptation based on Lidar, factors such as snowflakes, fog, and raindrops would introduce noise, which interferes with valid environmental point clouds, resulting in a significant decline in performance. Currently, datasets for severe weather conditions are limited. Enhancing the robustness of 3D detection algorithms to severe weather is imperative. This paper focuses on domain adaptation from clear weather serving as the source domain to snowy weather serving as the target domain where snowy labels are unavailable. The pri-mary strategies for domain adaptation include self-training [1], mean teacher [4], and loss consistency [5]. ST3D++ [6] serves as an advanced domain adaptation algorithm for cross datasets and sensors domains. We propose a weather domain adaptation paradigm named PDAST (point density-aware self-training), particularly from clear to snowy weather.