Drone Video Object Detection using Convolutional Neural Networks with Time Domain Motion Features

The drone video objection detection is challenging owing to the appearance deterioration, object occlusion and motion blur in video frames, which are caused by the object motion, the camera motion, and the mixture of the object motion and the camera motion in the drone video. One of the typical solutions is to use Convolutional Neural Networks (CNNs) to train detection model by taking single frame as input. The state-of-the-art method only uses the spatial feature of the single frame in the video, but makes no use of the motion features in the time domain. In this paper, we propose a method for detecting drone video object by using convolutional neural networks in combination with time domain motion features. The proposed method includes the steps of firstly extracting the motion information between the two neighbor frames, and then combining the extracted motion information with the baseline network. In the VisDrone2019 dataset, our proposed method is shown to be effective in detecting the objects, especially in significantly reducing the occurrence of the false and missed detection of objects.

I. Introduction

With the development and advancement of science and technology, drones (or Unmanned Aerial Vehicle UAVs) equipped with cameras have been widely used in many technical fields, including public safety, surveillance, aerial photography, environmental monitoring [1], etc. Generally, the detection of the object in the drone video plays a fundamental role in object tracking, object classification, event detection and behavioral analysis [2]–[8]. However, owing to the fact that the drone video contains the object motion, the camera motion, and the mixture of object motion and camera motion, there often incurs the problems such as video jitter, undesired change in object position, motion blur, appearance deterioration, and object occlusion, etc. Therefore, the accurate detection of object in the drone video is a challenging task in practical applications. Figure 1 illustratively shows motion blur, video defocus, and appearance deterioration in the drone environment. Fig. 1

Motion blur, video defocus, and appearance deterioration in existing drone videos