Towards emotionally-personalized computing: Dynamic prediction of student mental states from self-manipulatory body movements

An emotionally-personalized computer that could empathize a student, learning through a tutorial or a software program, would be an excellent application of affective computing. Towards development of this potentially beneficial technology, we describe two related evaluations of a student mental state prediction model that not only predicts student's mental state from his/her visually observable behavior but also detects his/her personality. In the first set of evaluations, we model the assumed cause-effect relationships between student's mental states and the body gestures using a two-layered dynamic Bayesian network (DBN). We used the data obtained earlier from four students, in a highly-contextualized interaction, i.e. students attending a classroom lecture. We train and test this DBN using data from each individual student. A maximum a posteriori classifier based on the DBN model gives an average accuracy of 87.6% over all four individual student cases. In the second set of evaluations, we extend the model to a three-layered DBN by including the personality attribute in the network, and then, we train the network using data from all four students. At test time, the network successfully detects the personality of each test student. The results demonstrate the feasibility of our approach.