A Novel Deep Multi-Task Learning to Sensing Student Engagement in E-Learning Environments

Automated sensing of the student's engagement in an e-learning system from emotional expressions remains a challenging problem due to varying conditions during the lecture. Such recognition and detection systems improve the teaching experience and efficiency by providing valuable feedback. Emotional expressions are expressed through non-verbal and verbal human emotional/behavior. More investigations are needed in this domain to carry out the learning process. Deep multi-task learning has been successfully employed in many real-world large-scale applications such as recognition systems. In this paper, we propose a novel education level state system to determine the student engagement level in an e-learning environment. The proposed approach is based on a hybrid deep multi-task learning technique. Soft and hard parameters are fused to achieve the best prediction. The performance of this system is evaluated on three facial expression benchmark datasets acquired in non-controlled environments. We validate the proposal using multi-input and mixed data to meet the relevant challenges.