Radar Trajectory-based Air-Writing Recognition using Temporal Convolutional Network

Air-writing systems offer users a virtual board to write characters or words in free space using fingers or hand movements. Several works have been proposed in literature that aim to use different sensors to enable such a system as an alternative to the keyboard and click form of human-machine interfaces. The advancement of miniature radar sensors and deep learning has enabled precise estimation and tracking of finger or marker movement followed by character recognition to offer an effective air-writing solution. However, deviating from earlier works in literature that make use of a network of radars to effectively track and recognize characters, in this paper, we propose to use only one or two radars to sense the local hand trajectory. We propose to use 1D temporal convolutional network (TCN) for simultaneous feature extraction and temporal modeling to recognize the drawn character from the local target trajectory. A dataset with 3750 character instances has been recorded using a 60-GHz millimeter-wave frequency-modulated continuous wave radar (FMCW) radar. We demonstrate the proposed end to end solution achieves a mean accuracy of 99.11% and 91.33% for two radar and one radar-based solution respectively outperforming other deep architectures.