Deep Skeleton Modeling and Hybrid Hand-crafted Cues over Physical Exercises

Human activities have always been complex and most important concern for researchers especially when it comes to physical exercises. Multiple methods have been proposed for physical exercise recognition using different sensors where the conventional approaches focused on either videos or motion-based sensors. Whereas, the combination of both types of data can improve the physical exercise recognition particularly for complex motion patterns. For that reason, a hybrid handcrafted cues-based method has been proposed in this paper. Data has been collected from the multi-modality-based datasets that are publicly available. Next, three different filters have been used to sift the noise from multiple sensors-based data. Then, an overlapping windowing technique along with human silhouette extraction has been utilized to pre-process the filtered data. Further, the hybrid hand-crafted cues have been extracted using linear prediction cepstral coefficients, Gaussian markov random field, and saliency maps. Finally, the cues have been reduced using multi-layer sequential forward selection methodology and the physical exercise activities have been classified using a deep belief network.