Virtual User Interface for Differently Abled People Using Vision Transformer Model

We are aware that using computers has become commonplace. But using digital devices like laptops, pcs and tablets is too challenging for those with varied abilities (those without hands and those who are paralyzed). Our goal is to use a webcam to read real-time video while integrating various face emotions with the corresponding mouse events. In order to imitate mouse motions and order, we used pyautoGUI in conjunction with OpenCV for image processing and computer vision tasks. Image processing is mainly used to identify faces and facial expressions. For key board related functions, we integrated speech to text conversion. Speech is transformed to text whenever user speaks and it will be displayed according to the user's wish like when the cursor is over a search box or a micro soft word. By doing this, all the computer tasks can be accomplished without the usage of hands.