PosEmotion - Combining Real-Time 2D Body Pose Estimation and Facial Emotion Recognition to Analyze Human Behavior

Facial emotion recognition and pose estimation have been topics of great research interest in the past, especially in the last two decades. While several methods have achieved commendable results in both domains, most of them have focused on the fidelity and accuracy of predictions. Moreover, we believe that combining facial emotion recognition with body pose/posture estimation provides a more comprehensive understanding of the actions and behavior of a person. We propose an end-to-end pipeline that combines both i) pose estimation and ii) facial emotion recognition to analyze human behaviour. We achieve real-time performance due to optimal network size and reduced number of parameters compared to existing techniques, with a minimal trade-off in accuracy. We train the networks separately and combine them at inference time to form an end-to-end pipeline. Our technique has a low inference time, achieves real-time performance without compromising in accuracy, which can be beneficial for real-world applications such as surveillance, remote supervision, and proctoring systems.