Machine Learning Aided Minimal Sensor based Hand Gesture Character Recognition

Hand gesture recognition is the process of detecting the hand movements via sensor measurements for detecting an activity, such as writing a letter or a number. Recognising the handwritten characters using wearable devices enables machine-human interaction to occur without the need for a communication method. An intelligent automated framework is required to accurately detect the handwritten characters using wrist worn sensor signals, in particular, with minimal number of sensors. Moreover, the system developed needs to have the capacity to recognise the characters written in different sizes. In order to address these, we analyse performance of several machine learning models using single/multiple sensors namely, accelerometer or/and gyroscope, for recognising hand gesture characters including alphabet and numbers of varying sizes. We formulate a set of features that enable robust and accurate detection of the characters.We performed novel data collection using an off-the-shelf wrist-worn sensor based device, and evaluated our framework to detect the different characters effectively. The maximum accuracy (90.40%) was achieved using both sensors and Random Forest (RF) model. This was dropped to 82.51% for the same model using accelerometer sensor alone. Using the gyroscope sensor, an overall average accuracy of 80.16% was achieved with the Forward Neural Network (FNN) model. Although the model based on both sensors showed the best performance, our evaluation reveals that it is feasible to develop a machine learning model using single sensor to detect hand gesture characters of varying sizes with reasonable (≥ 80%) accuracy.