Comparing YOLO-based Detectors for Pedestrian and Car Detection in Aerial Static Video: An Evaluation of Generalization Capacity and Performance

Over the last decades, the famous deep-learning framework of ‘You Only Look Once’ – YOLO for object detection has gasped important interest due to its vital performance for object detection. In this paper, we compare three deep learning architectures of YOLO family for the task of pedestrian and car detection from static video frames. We have chosen YOLOv5s, YOLOv5su, and YOLOv8s architectures as they can perform real-time detection, which is the focus of our study. Our training and evaluation were conducted on a dataset from Stanford University, which includes a combination of eight birds-eye view scenes. The dataset was separated into train, validation, and test sets, and the discrepancies between validation and test results were used to evaluate the generalization capacity of each architecture. For the purpose of evaluating the generalization capacity of the model, we isolated one scene as the test set, ensuring it was not seen during the training phase. Our experimental results show that the recent YOLOv5su and YOLOv8s do not outperform YOLOv5s in terms of accuracy and generalization despite their anchorless architecture. The challenges faced by all the architectures in terms of generalization can be ascribed to a combination of factors, namely the model's capability to differentiate the background and the static characteristics of the dataset frames.

I. Introduction

In the realm of object detection from unmanned aerial aircraft, the integration of artificial intelligence (AI) has brought about significant advancements with profound implications for security. AI-powered algorithms have revolutionized the capabilities of aerial surveillance systems, enabling real-time identification and tracking of pedestrians and vehicles from elevated perspectives. This innovative fusion of AI and aerial platforms enhances security measures in various critical contexts, such as border control, disaster response, and law enforcement. AI algorithms can discern suspicious activities, track unauthorized movements, and even predict potential security threats by analyzing patterns and anomalies in the data collected from these unmanned aircraft. The incorporation of AI into the surveillance process not only improves the accuracy and efficiency of detection but also extends the reach of security operations, enhancing situational awareness and response capabilities, ultimately contributing to more robust security strategies in today's dynamic and evolving security landscape. To do so we need to ensure serverless computing, edge computing and secure clouds, to deliver an open-source, scalable, efficient, and trusted solution, able to seamlessly operate on core and edge cloud infrastructures for time-critical, self-hosted applications, in a European, privacy-preserving, green and responsible data-centric model. We intent to deliver our models through TRUSTEE [1] that enables the delivery of a comprehensive array of information services to various stakeholders, ranging from citizens to professionals, the platform integrates and amalgamates existing specialized knowledge alongside services relevant to the BORDERUAS project [2].