Lidar and Camera Heuristic Learning Method for Target Detection

Lidar and camera are crucial to intelligent mobile platforms to perceive environment. Generally massive data are annotated manually to training deep neural networks for target detection. However, manual labelling of large amounts of sample data would cause exorbitant costs and inefficiencies. To improve the learning efficiency, a Lidar and camera heuristic learning method is proposed for target detection of mobile platforms. Considering the different detection capabilities of Lidar and camera for different features in different scenes, the detection results from different sensors are used to train another sensor respectively to improve the detection models. Firstly, targets are detected from real-time sensor data by pre-trained models associated with Lidar and camera separately. Then the semantic relevance of detection results are evaluated by data association to obtain association matrix and consistency vector. Finally, detection models of Lidar and Camera are trained with associated detection results respectively to improve performance continuously. The experimental results show that the workload associated with the manual labeling of different detection samples is reduced effectively, and the training efficiency of target detection models is improved greatly.