Object based Human-Object Interaction (HOI) Recognition using Wrist-mounted Sensors

Detecting and recognizing objects with human interaction is very essential for wide-ranging applications, such as health monitoring, human behavior analysis, and access control. In the last few years, numerous approaches have been introduced for object detection and activity recognition using the image and video processing-based techniques. However, these approaches are exposed to privacy challenges and are computationally expensive. The ubiquitous sensing systems based on wearable inertial sensors, such as the accelerometer and gyroscope, provide an effective solution to these hurdles. Thus, in this research work, we present an approach for recognizing objects and human-object interactions based on wearable inertial sensors. In this aspect, we selected three objects with fourteen different interactions and recorded data from 15 different participants using sensors embedded in the commercially available wearable wristwatch (Metawear). In the first step, we extracted different time-domain features and performed object classification. Next, we undertook the object-dependent interaction recognition for classifying object interactions. For testing recognition results, we used three different classifiers (random forest (RF), k-nearest neighbors (KNN), and Naive Bayes (NB)), and achieved the best accuracy rate of 87%, 86.33%, 87.22% using RF classifier for laptop, mobile, and notebook, respectively.