Inference of Student Needs in an Online Learning Environment Based on Facial Expression

Due to the limited interaction between students and teachers in online environments, teachers often struggle to keep track of their students' needs in terms of teaching response. The purpose of this research is to construct a model that infers student needs based on their facial expressions while they are learning online. An experiment collected video recordings of students' faces while learning and a survey collected their reported needs. A neural network model was created to infer the reported needs from facial expression data extracted from the videos as action units in Facial Action Coding System (FACS). A neural network was trained to infer the student need responses from the factors obtained through Factor Analysis. Inference model testing demonstrated that the model correctly identifies reported needs 87% of the time on video samples not used for training. The results suggest this approach may contribute to the development of improved online learning systems that will allow teachers to understand in real-time how students want them to respond.