Self-driving Deep Learning System based on Depth Image Based Rendering and LiDAR Point Cloud

We present a “Self-driving deep learning system based on depth image based rendering and LiDAR point clouds” in this paper. Using the depth image based rendering (DIBR) to obtain complete point clouds, combining with the color image modified by histogram equalization and optimal profile compression (HEOPC), and further using MobileNet-YOLO neural network to achieve accurate 3D environment detection and identification. Using the DIBR to generate parallax map information, and obtaining the depth image through the conversion formula. Then combining LiDAR point cloud and the depth image to repair the objects in the point cloud image. We combine HEOPC and the accuracy of deep learning to optimize the color image enhancement of the night environment. According to the restored point cloud information, we perform a cutting algorithm to divide the area of interest. Then through MobileNet-YOLO neural network training, to identify the area of interest, including people, cars and bus.