A Deep Learning Approach for Drone Detection and Classification Using Radar and Camera Sensor Fusion

With the growth of Unmanned Aircraft Systems (UAS) technology and the increasing misuse of small UAS (sUAS), the importance of a reliable method for detecting and classifying aircraft from other flying objects has become apparent. The current approaches for detecting and classifying aircraft and other flying objects are primarily based on solutions that rely on a single sensor, either visual data features or micro-Doppler extraction from radar data. However, these methods may have limitations when it comes to detecting objects at greater distances or in challenging weather conditions. To address the problem, the paper proposes a joint classification network based on radar and camera fusion. The radar network extracts the Spatio temporal features from the radar track and the camera network extracts the deep, complex features from the image. A synchronized radar and camera data is established using multiple field trials during different times of the year. The radar classification using a combination of IMM filters and RNN, the camera detection and classification using YOLOv5, and the combined joint classification network are evaluated on the field dataset. The experimental results greatly increase the classification performance for drones and birds, respectively, to 98% and 94%. This is especially true in situations when a single sensor would struggle to offer reliable classification. The system can accurately classify drones while reducing false alarms caused by other objects, such as birds.