Self-driving simulation scene reconstruction using self-supervised depth completion

The rapid development and testing of autonomous driving technology heavily rely on several complex and diverse simulation scenarios. Nowadays, the most common way to generate simulation scenes is using computer graphic techniques or game engines. However, this method requires manual creation of various on-scene elements, such as vehicles, pedestrians, traffic signs. Moreover, material properties, have limited scalability, and require a lot of time and effort. In this paper, we propose a simple and effective method to reconstruct realistic self-driving scenes, more realistic and accurate than other methods that generate scenes using real data. Our method uses self-supervised depth completion to fuse multimodal information from image and LiDAR data, preserving both the rich textures and colors in the scene while retaining accurate distances and shapes. Furthermore, we provide valid values for places in LiDAR with no data by enriching and enhancing the detail and realism of the reconstruction results. Additionally, our depth completion neural network is also self-supervised, which avoids the difficulty of labeling large data. Extensive experiments on the KITTI urban and Cross-country datasets show that our method outperforms other state-of-the-art methods in both depth completion accuracy and scene reconstruction effect.