Temporal-Dependent Features Based Inter-Action Transition State Recognition for Eldercare System

Elderly individuals are particularly vulnerable to accidents, with a significant number of incidents occurring during transition states between primitive actions such as sitting to standing and sitting to lying. This paper introduces a novel machine-learning technique in artificial intelligence, based on temporal-dependent features to assist the elderly. To ensure privacy, we employed stereo depth cameras for data acquisition from the elder care center and exclusively processed depth images. The first step of our approach involves localizing individuals using the YOLOv5 object detector. Subsequently, we employed the Segment Anything Model to segment only the person masks, excluding other areas from consideration. Temporal-dependent features were then extracted for every five frames from the subsequent person masks that enable the recognition of transition states from primitive actions. We tested various classification approaches and compared the results by defining norms and metrics. Our experimental findings demonstrated that the overall accuracy rates for classifying 2 classes and 5 classes on small segments are 91.18% and 91.67% respectively. To validate the effectiveness of our proposed method, we conducted experiments using real-life environments inside three rooms and obtained average accuracy rates of 90.17%, 97.16%, and 77.44% respectively. Overall, this model has the potential to enhance the safety and well-being of the elderly population.