Efficient Real-time Fall Prediction and Detection using Privacy-Centric Vision-based Human Pose Estimation on the Xilinx® Kria™ K26 SOM

Falls among elderly people can lead to serious injuries and significantly impact their quality of life. This research proposes a portable, low-power battery-operated, vision-based fall prediction and detection system using a human pose estimation (HPE) model running on an edge computing device, the Xilinx® Kira™ K26 System-on-Module (SOM). The system consists of an Intel® RealSense™ D455 range-sensing camera connected to a KV260 platform. The camera captures synchronized RGB and depth frames of pixel dimensions 640x480x3 (HxWxC) and 640x480, respectively, at a rate of 60 frames per second (FPS) for real-time processing. The KV260 employs a 3-stage pipeline architecture to process the frames through quantized YOLOX, A2J, and fall detection machine learning models.The YOLOX model takes each RGB frame with a dimension of 640x640x3 to produce a bounding box for each human in the frame. The system then discards the RGB frame, preserving privacy. The second model, Anchor-to-Joint (A2J), accepts a depth frame with a dimension of 288x288. The model then produces 15 key points of joint information for each detected human. The final model is a convolutional neural network (CNN) that takes informative joint coordinates (x, y, z) from a set of frames and predicts and detects human fall activity. The fall classification output is displayed on an LCD display.The YOLOX model has a quantized accuracy of 74% and an inference time of 43ms. The A2J model has a quantized accuracy of 76% and an average inference time of 49ms on the SOM board. The fall detection model has a quantized accuracy of 72.86% and an average inference time of 1.17 ms.