Concurrent Activity Classification and Human Identification Based on Point Cloud Data and Deep Learning

Human sensing has long been an actively studied problem due to its potential in a wide range of applications. However, most existing approaches depend on cameras that require proper light condition and also pose quite a challenge for privacy protection due to native nature of easy understanding by eyes in the visual data. As a promising alternative, mmWave radars have been explored recently as a viable alternative to provide precise human sensing capability without aforementioned concerns. In this paper, we propose to use commercial low-cost mmWave radars to perform multiple human sensing functionalities concurrently and accurately based on a multi-task learning architecture. We show that the proposed system is able to classify activities and identify human individuals at the same time such that question of “Who is doing what” can be asked. Our experiments show that an average accuracy of 98.4% and 97.6% are achieved respectively for the two tasks by the proposed method, implying its great potential for effective human sensing using mmWave radars.