Real-Time Object Geopositioning from Monocular Target Detection/Tracking for Aerial Cinematography

In recent years, the field of automated aerial cinematography has seen a significant increase in demand for real-time 3D target geopositioning for motion and shot planning. To this end, many of the existing cinematography plans require the use of complex sensors that need to be equipped on the subject or rely on external motion systems. This work addresses this problem by combining monocular visual target detection and tracking with a simple ground intersection model. Under the assumption that the targets to be filmed typically stand on the ground, 3D target localization is achieved by estimating the direction and the norm of the look-at vector. The proposed algorithm employs an error estimation model that accounts for the error in detecting the bounding box, the height estimation errors, and the uncertainties of the pitch and yaw angles. This algorithm has been fully implemented in a heavy-lifting aerial cinematography hexacopter, and its performance has been evaluated through experimental flights. Results show that typical errors are within 5 meters of absolute distance and 3 degrees of angular error for distances to the target of around 100 meters.

I. Introduction

Aerial cinematography has become an increasingly popular tool for capturing footage in a variety of media production contexts, including sports events, commercials, and movies. While manual operation offers a high degree of artistic control and the ability to achieve professional-looking shots, recent advancements in autonomous technology have made it possible to replicate these results through automated means. The literature has identified and described visually pleasing combinations of framing shot types and camera movements [1]. A common thread among these descriptions is the need for knowledge of the target's 3D position in a world coordinate system at all times. This requires the use of a target detection, localization, and tracking system, particularly in unstructured environments. It must be able to run in real-time, onboard the UAV, and should not rely on external sensors. Recent advances in computer vision techniques have emerged as a highly efficient means of performing object detection and tracking. By combining this technology with a pinhole camera model and a fast 3D position estimation algorithm, a highly efficient real-time object localization tool can be created for short and medium-distance UAV cinematography and aerial shot planning.