Derivation of Geometrically and Semantically Annotated UAV Datasets at Large Scales from 3D City Models

While in high demand for the development of deep learning approaches, extensive datasets of annotated unmanned aerial vehicle (UAV) imagery are still scarce today. Manual annotation, however, is time-consuming and, thus, has limited the potential for creating large-scale datasets. We tackle this challenge by presenting a procedure for the automatic creation of simulated UAV image sequences in urban areas and pixel-level annotations from publicly available data sources. We synthesize photo-realistic UAV imagery from Google Earth Studio and derive annotations from an open CityGML model that not only provides geometric but also semantic information. The first dataset we exemplarily created using our approach contains 144 000 images of Berlin, Germany, with four types of annotations, namely semantic labels as well as depth, surface normals, and edge maps. Based on this specific case, we demonstrate the entire pipeline and give a comprehensive overview of technical obstacles and solutions. In experiments, we evaluated the quality of our dataset and its potential application in monocular depth estimation and semantic segmentation using deep learning methods. To facilitate the creation of further datasets, we provide our source code along with the produced dataset at https://github.com/sian1995/largescaleuavdataset.