Multimodal Fusion-GMM based Gesture Recognition for Smart Home by WiFi Sensing

With the thriving development of communication systems and the ubiquity of commodity WiFi devices, wireless intelligent sensing has attracted increasingly attention owing to its importance in Human-Computer Interaction (HCI) where hand gesture recognition has been widely studied these years. However, most of gesture recognition approaches have a limited performance owing to the background noises. In this paper, we present a multimodal fusion-Gaussian mixed model (GMM) based gesture recognition scheme by exploiting channel state information (CSI) of WiFi signals. To this end, we firstly design two theoretical underpinnings, including a sensing model and a recognition model. Firstly, the sensing model is established to investigate the impact of gestures on the propagation properties of WiFi signals and quantify the correlation between the CSI dynamics and various gestures. Secondly, the recognition model is used to exploit physical gesture-induced signal changes to infer potential gestures information. Then, we implement the proposed scheme on a set of WiFi devices and evaluate it in both laboratory and corridor environments. The experiment results show that the proposed scheme can achieve average recognition accuracies of 96% and 94% in these two scenarios, respectively. This shows promise for future ubiquitous hands-free gesture-based interaction with mobile devices.