Estimating Object Perception Performance in Aerial Imagery Using a Bayesian Approach

In this work, we present a novel approach for predicting the perception performance of an optical aerial surveillance system. We outline our use case in multi-target tracking motivating this work before giving some background on related work in sensor performance estimation. After explaining the reasoning behind selecting a Bayesian network against other machine learning classifiers, we present relevant underlying concepts of sample-based inference and parameter learning in probabilistic graphical models. We present the modular architecture of our performance estimator and reason through the model input nodes based on system, object, and scene parameters. Further, we describe the extensive flight experiment performed using a crewed ultralight aircraft equipped with a state-of-the-art aerial surveillance gimbal to collect imagery of target vehicles under carefully specified conditions. We describe the implemented post-processing of the recorded data, the method used to sample a reasonably sized dataset for learning, the criteria followed while labeling the dataset, and the technique used to learn a usable model based on a Bayesian network. Next, we analyze the classification performance of the model on an independent validation dataset and compare it against two additional machine-learning classifiers trained on the same dataset. We discuss the limitations of the proposed model and the presented validation. We wrap up by giving an outlook on various opportunities for possible future improvements to enhance the prediction performance of our network. Finally, we present other use cases benefitting from a similarly structured sensor performance prediction model, such as search planning.