Computer Vision based Student Behavioral Tracking and Analysis using Deep Learning

Over the past couple years, The Covid-19 virus has triggered a global pandemic, requiring us all to remain isolated in our homes. Students and professionals were forced to move to the virtual world to resume their academic activities. This has led people to use video conferencing applications on a massive scale. However, it has been a challenge to monitor many students at the same time by a single invigilator/faculty thereby, the student engagement and interaction has dropped in a rapid scale. To overcome this challenge, an open-sourced modular system is proposed to Face identification for face spoofing using CNN and students’ gaze tracking with head positioning & eye tracking in real-time virtual video conferencing applications using computer-vision & deep-learning. With a single-Shot-Multibox detector, using ResNet-10 Architecture as the foundation, the student’s face is recognized and detected. Then to take facial landmarks, pre-trained Histogram Oriented Gradients (HOG) and linear SVM object detectors are used. With the captured features, the eyes are detected using dlib and the centers of eyeballs are extracted using OpenCV eye classifiers. This paves the way to perform eye tracking and gaze tracking. All the stated Machine learning modules’ output are aggregated and translated to a graphical representation as the final analysis of the students’ behavior in the virtual sessions.