Temporal-Range-Doppler Features Interpretation and Recognition of Hand Gestures Using mmW FMCW Radar Sensors

This paper introduced a comparative study of using deep neural networks in non-contact hand gesture recognition based on millimeter wave FMCW radar. Range-doppler maps are processed with a zero-filling strategy to boost the range and velocity information of gesture motions. Two optimal types of deep neural networks, 3D-CNN and CNN-LSTM are respectively constructed to reveal the temporal gesture motion signatures encoded in multiple adjacent radar chirps. With the proposed networks, the recognition accuracy of six popular hand gestures reach to 95%. Meanwhile, this letter further explores the performance of the proposed networks in the impacts of training data size on the recognition accuracy. The proposed methods can be applied in the recognition of minor finger motions, providing some preliminary experimental results compared with other baseline methods.

I. Introduction

Recently, many efforts in radar-based sensing have demon- strated the capacity of real-time or near-real-time high resolution imaging using a millimeter wave (mmW) band FMCW radar [1] [2] [3] . Currently, convolutional neural network (CNN) is a popular high efficiency algorithm with extensive applications in image and speech processing. The contribution of CN- N is locally perceiving and learning the optimal features on its own. Due to the advantages over traditional machine learning algorithms such as Support Vector Machine (SVM), Decision Tree, and Back-propagation network, CNN can provide a new solution of non-contact hand gesture recognition using millimeter wave (mmW) Frequency Modulated Continuous Wave (FMCW) radars by avoiding the complicated pre-processing and directly utilizing the original data as the input. In 2016, Google published its Soli project to distinguish complex finger movements and to deform hand shapes by applying deep learning as the classifier in gesture recognition [4] , [5] . Ref [6] also introduces a joint CNN application in gesture recognition with a recognition accuracy of 96%. However, Soli project utilizes a specially customized 60 GHz FMCW radar with high temporal resolution. Ref [6] [7] [8] only identifies target gestures with arm motion, and the recognition of tiny but also frequently used finger gestures is not involved. To implement the recognition of more tiny gestures based on a commercial mmW FMCW radar, this letter proposes two types of CNN structures and evaluates their performance in training time, network complexity, and the impacts of training data size on the recognition accuracy in different backgrounds.