Implementation of a Novel Framework for Physically Disabled Patients using Human-Computer Interface

Human-Computer Interface (HCI) refers to how a user interacts with a computer through the use of various input/output devices. Computers are now capable of seeing the world from a perspective similar to humans due to current developments in the fields of Computer Vision and Object recognition and tracking. Users automatically move their heads in the direction of their field of vision. Using the biological feature of the head's autonomous and involuntary movement, any facial expression can confirm what the user wishes to focus on and agitate the mouse cursor. The technology was created intends to address several shortcomings of present interface devices. This study extracts the user's images from the video stream using a webcam and processes them in real time to convert them into cursor motions. Physically disabled patients' datasets are collected, preprocessed using the Modified Histogram Equalization, and features are extracted using the Gabor filter bank. A bounding box is created by ResNet 101 around the recognized face. Enhanced Multi-Label Sequential convolutional neural network (EM-LSCNN) is proposed to recognize the facial landmarks of the provided face. When the system is implemented and fine-tuned following the user, it will change the way the mouse indicator moves across the screen's viewport, completely doing away with the need for a mouse. The proposed model performed with the greatest accuracy of 98.85%, a precision of 99.20%, a recall of 98.30%, and an f1-score of 98.65% when compared to existing approaches.