RS-Aug: Improve 3D Object Detection on LiDAR With Realistic Simulator Based Data Augmentation

Light detection and ranging (LiDAR) is an essential sensor for three dimensional (3D) object detection via generating 3D point cloud of the surroundings, and it has been widely used in the various visual applications, especially autonomous driving. However, limited numbers of labeled LiDAR datasets brutally restrain the development of 3D object detector, and this situation breeds an urgent demand on data augmentation in this field. By far, most of the traditional methods reuse the labeled samples, while those unlabeled are hastily untaken. Motivated by this, we propose a
R
ealistic
S
imulator based data augmentation (RS-Aug). It aims to construct augmented real scenes to enrich the diversity of training dataset. To train 3D object detector in a supervised learning way, the first step of RS-Aug is auto-annotation. Time-continuous LiDAR frames are used to construct the dense scene, which is beneficial to annotation and the subsequent rendering augmentation. However, 3D points with incorrect semantic labels are naturally gathered during multi-view reconstruction, causing the negative effect on auto-annotation. We propose an algorithm of cluster guided
$k$
-nearest neighbor (c-
$k$
NN). It emphasizes on de-nosing semantic labels of clustered points using distance and intensity constraints. Then, the next step of RS-Aug is rendering augmentation on the real scene. To enhance the rendering quality using collision and distance constraints with the less computation complexity, we propose a scheme of heuristic search (HS) based object insertion. It estimates the proper position of the inserted object from 2D bird’s eye view (BEV). Experiments demonstrate the de-noising accuracy of c-
$k$
NN, rendering quality of HS based object insertion, and improvement of RS-Aug on object detection.