Gesture-mmWAVE: Compact and Accurate Millimeter-Wave Radar-Based Dynamic Gesture Recognition for Embedded Devices

Dynamic gesture recognition using millimeter-wave radar is a promising contactless mode of human–computer interaction with wide-ranging applications in various fields, such as intelligent homes, automatic driving, and sign language translation. However, the existing models have too many parameters and are unsuitable for embedded devices. To address this issue, we propose a dynamic gesture recognition method (named “Gesture-mmWAVE”) using millimeter-wave radar based on the multilevel feature fusion (MLFF) and transformer model. We first arrange each frame of the original echo collected by the frequency-modulated continuously modulated millimeter-wave radar in the Chirps × Samples format. Then, we use a 2-D fast Fourier transform to obtain the range-time map and Doppler-time map of gestures while improving the echo signal-to-noise ratio by coherent accumulation. Furthermore, we build an MLFF-transformer network for dynamic gesture recognition. The MLFF-transformer network comprises an MLFF module and a transformer module. The MLFF module employs the residual strategies to fuse the shallow, middle, and deep features and reduce the parameter size of the model using depthwise-separable convolution. The transformer module captures the global features of dynamic gestures and focuses on essential features using the multihead attention mechanism. The experimental results demonstrate that our proposed model achieves an average recognition accuracy of 99.11% on a dataset with 10% random interference. The scale of the proposed model is only 0.42M, which is 25% of that of the MobileNet V3-samll model. Thus, this method has excellent potential for application in embedded devices due to its small parameter size and high recognition accuracy.

I. Introduction

Hand gestures are a great way to interact with computers and have many potential applications, including smart homes, self-driving cars, and sign language translation [1], [2], [3]. Nevertheless, before we can do any of that, we need to accurately recognize the hand gestures we previously set as necessary for the computer to understand.