A Smart Virtual Tutor with Facial Emotion Recognition for Online Learning

The main aim of our work is to determine whether a virtual tutor can effectively deal with student queries in online distance learning environments. We adopted a Machine Learning (ML) approach to devise a prototype of a smart autonomous tutor which can understand queries and provide responses. Furthermore, with the use of the Web Camera, the virtual tutor can identify the emotion of the student when reading the provided response. This helps the virtual tutor to assess the appropriateness of the provided response. Students can type or use voice commands to ask a question and the response is also provided in both text and voice. The approach is mainly based on the application of the Recurrent Neural Network (RNN) with the Long Short Term Memory Algorithm (LSTM) to determine the query of the user. Emotion detection is based on the Convolutional Neural Network (CNN) approach. Our experiments demonstrate testing accuracy of around 87% for query detection and around 62% for emotion detection.