SpeakingEyes: Enabling Paralyzed People to Communicate

SpeakingEyes is a Human-Computer Interaction (HCI) based desktop application. It enables people with paralysis, locked-in syndrome, communication disorders, Lou Gehrig's disease, spinal cord injuries, neurological disorders and related disabilities, who are unable to perform the voluntary movement of muscles, to express themselves and perform essential activities independently via eye gaze. These include communicating with people using onscreen keyboard, requesting various services such as food, water, natural calls, phone calls, asking for assistance, raising alarm during an emergency, turning on/off switches for lights, fan, and other home appliances. The eye gaze is determined by performing image processing on image of user's eyes. Web-camera/external camera is used to acquire image of the user's eyes. After preprocessing it is fed into Convolutional Neural Network (CNN) which is used as a classifier to predict eye gaze. This CNN was trained on a custom dataset and gives an overall accuracy of 98.89% with an average processing time of 0.6 seconds per frame on Intel i5 CPU. After the prediction, the subsequent action is taken by the computer.