Multi-model Deep Learning Drone Detection and Tracking in Complex Background Conditions

The recent popularity of drones with quadcopter layouts is threatening public safety and personal privacy. With the ability to hover and perform complex maneuvers even in indoor conditions, equipped with video cameras as well as capable of carrying hazardous materials, drones can truly become a security threat, especially to vulnerable organizations. Therefore, detecting and tracking drones in secured areas poses an urgent task for the surveillance system. In this paper, we design a real-time drone detection and tracking system with the combination of multiple deep learning and computer vision techniques: 1) Yolo-v4 model for detecting drones and 2) visual models for tracking drones. Besides, we have collected and labeled a larger drone dataset by mixing the existing datasets with our collected images. We evaluated three deep learning models for drone detection on this dataset and acquired the Yolo-V4 model to be the highest detection performance with AP = 34.63%. Combining this detection model and the existing visual tracking modules can boost the drone tracking up to more than 20fps for different backgrounds at around 700m by using an usual PC without GPU.