Multi-HAR: Human Activity Recognition in Multi-person Scenes Based on mmWave Sensing

Human activity recognition (HAR) holds a wide range of promise in many domains, such as health monitoring, human-computer interaction, and intelligent industry. Recently, millimeter-wave (mmWave) has become a hot topic of research in the field of HAR due to its fine-grained sensing capabilities. Not only that, mmWave allows HAR to be implemented in a contactless and privacy-protective way, avoiding the hassle associated with visual sensors and wearable devices. However, most of the existing research has only explored HAR in single-person scenes. To explore the feasibility of HAR in multi-person scenes, we propose Multi-HAR, a system with high accuracy based on Group Tracking Algorithm, 3D-CNN, and LSTM. The Group Tracking Algorithm succeeds in accurately clustering and tracking each target in a multi-person scene and assigning the point cloud to the target it belongs to, which is an important prerequisite for the implementation of Multi-HAR. We test Multi-HAR offline, using a dataset collected by ourselves containing five common human daily activities: sitting, walking, standing, falling, and bending, and achieved an average accuracy of 90.7%, 86.2%, 82.2%, and 84.0% in single-, two-, and three-person, and mixed scenes, respectively. We develop a real-time system using Multi-HAR, which also performs satisfactorily in practice.