A Multi-view 3D Vehicle Detection Method Based On Novel 3D Proposal Generation Method

This paper introduces a multi-level fusion 3D vehicle detection method based on novel 3D proposal generation method for autonomous driving scenes. The entire detection network is composed of two subnets: a region proposal network (RPN) and a final detector network. The RPN is designed to generate reliable 3D object proposals for multiple targets using pixel-level fusion of depth map from lidar point cloud and images from camera processed by a 2D detection structure. To further improve the accuracy of 3D object proposals, a novel multi-source fusion based depth prediction method is proposed, which outputs the depth from the target while detecting 2D bounding boxes. The final detector network processes the 3D proposals generated by RPN to extract multi-view features, which serve as input to predict the 3D location, orientation and classification of targets. The whole detection method cleverly realizes the information fusion of heterogeneous data from multiple sensors at multiple levels. The experimental results on the challenging KITTI dataset prove that our method has certain advantages compared with the existing multi-view fusion detection methods.