Early Gesture Recognition With Reliable Accuracy Based on High-Resolution IoT Radar Sensors

Early recognition of gestures gives a better user experience during human–computer interaction in a near real-time way. However, the inadequate information accumulated in the early stages of the gesture will make the machine ambiguous and respond wrongly. Therefore, the challenge of early recognition lies in choosing the right time to trigger gesture prediction and using limited information to achieve reliable accuracy. In this article, an early gesture recognition method is proposed to achieve almost the same accuracy as a complete gesture sequence. For this purpose, we design a new architecture to modify the current feature sequence according to the existing information and introduce a new loss function to maximize the probability of correct gesture label as early as possible. At the same time, gesture recognition is triggered when the maximum probability of model output is greater than a set threshold. The publicly available data set used for evaluation comes from Google’s Project Soli sensor. Experimental results show that the proposed method for early recognition can achieve a gesture recognition rate of 88.85% with an average time saving of 69.47%. Compared with the results of all sequence recognition, the recognition rate drops by only 1.7%. In addition, the proposed method based on the data set constructed by terahertz radar also shows similar early recognition performance and can be applied to other radar sensors.