Drone-based Generation of Sensor Reference and Training Data for Highly Automated Vehicles

Highly automated vehicles require complex sensor systems to accurately perceive their environment and safely reach a destination. To achieve the necessary perception quality, these sensor systems typically consist of a combination of several sensor types. Due to the resulting complexity in hardware and software, it is necessary to use large amounts of highly accurate reference data for evaluation. Previous approaches to generate these data (e.g. RTK-GNSS-INS) are limited regarding the applicability in real traffic or the achievable accuracy. We present a drone-based approach to circumvent these problems. A drone follows a sensor system under test on a test vehicle in real traffic and captures its surroundings using a camera. After spatio-temporal synchronization, reference data can be generated from the camera imagery using deep learning-based object detection and tracking. We show that this approach and the quality of the derived data not only allows the evaluation of a LiDAR-based road user detection system, but can also be used to improve the system if used as training data. Within the scope of the conducted experiments, the average precision for cars was improved by 30 percentage points, which resulted in the false negative rate reduction by more than 60% for midrange distances on a selected route.