A Lightweight Gesture Recognition Method on Ultralow-Power Nano-UAVs

Nano-scale unmanned aerial vehicles (UAVs) are a kind of aircraft with extremely small size (sub ten centimeters) and weight (a few tens of grams). It is challenging for resource constrained nano-UAVs to be equipped with intelligent human-drone interaction(HDI) functions such as gesture recognition based on artificial intelligence methods. Also, nano-UAVs can usually only carry low resolution gray-scale cameras due to the constrained onboard resource, which raise the difficulty of gesture recognition. This work attacks a complex task to enable the nano-drones with function of HDI ability. A lightweight deep learning method is proposed to realize static gesture recognition on resources severely limited nano-drones. This paper presents a vertically complete approach starting from the convolutional neural network (CNN) model design, model training, dataset collection and dataset augmentation down to 16-bit quantization, and onboard deployment. The proposed method achieves a recognition accuracy of more than 85% while meeting the real-time requirements. Even if the drones work in the energy-saving mode, the method can also achieve a speed of more than 6FPS. The data collection, augmentation and model training strategies for nano-drones improved the generalization and robustness of the model. This work also provides a solution for other similar human-drone interaction tasks on nano-drones.