Adaptive Body Part ROI with Triple Stream Approach for Human Action Recognition

This study explores Adaptive Regions of Interest (ROI) methods in Human Action Recognition (HAR) through the utilization of OpenPose keypoints for ROI image generation from video data. Utilizing the NTU RGB+D 60 dataset and the EfficientNetB7 model, we examine ROIs ranging from full-body to specific joint segmentations. We propose a Triple Stream approach-where each stream employs a unique ROI image generation process. Our results demonstrate that the Triple Stream approach, combining Full Body Segmentation, 7 Joint ROI, and 6 Joint ROI, significantly enhances HAR accuracy for the XSUB benchmark. Similarly, for the XVIEW benchmark, a combination of Full Body Segmentation, 7 Joint ROI, and 3 Joint ROI significantly improves accuracy. Our proposed approach can also be adapted to enhance the performance of other models. Notably, by integrating the Triple Stream approach with alterations to the RGB channel in MMNet [20], we achieve accuracies of 97.2% on the XSUB benchmark and 99.3% on XVIEW.