Integrating Natural Language Processing & Computer Vision into an Interactive Learning Platform

Virtual learning platforms are digital interfaces that allow educators and learners to communicate and share resources to facilitate learning. Currently, many of them fail to retain engagement amongst a wide demographic of students. Barriers such as language differences and poor digital literacy often make platforms difficult to use. The purpose of this research was to create a comprehensive learning platform by incorporating Natural Language Processing (NLP) and Computer Vision (CV) to promote an improved user experience through human-computer interactions and graphics. The platform was developed in Python using the Google Speech Recognition API for NLP and the OpenCV library for image processing. After integrating these features into the socket server platform, they were rigorously tested for accuracy to assess the success of a more comprehensive learning environment.

I. Introduction & Problem Definition

Recently, virtual learning has become increasingly popular as the reliance on technology makes its way into educational institutions. Platforms such as Zoom, WebEx, and Google Classroom have been growing in usage for their ability for lecturing and assignment distribution. According to the National Center for Educational Statistics, 3.3 million students attended college exclusively online in 2018 [1]. Online learning platforms allow for a more tailored educational experience by providing disadvantaged students with convenient resources that would be otherwise inaccessible due to geographic or socioeconomic barriers. As a result, virtual learning is now a serious educational option for both private and public institutions.