Mobile Application based Sign Language System for Hearing and Speech Impaired People

The primary obstacle faced by individuals who have hearing or speech impairments when interacting with others is communication. Successful communication necessitates the participation of all participants. Learning sign language isn’t easy, and developing and practicing conversation skills takes time. This imposes a problem for the hearing and speech impaired people for communicating with others. Hence, this project aims to create a mobile app that facilitates bi-directional communication among individuals with hearing and speech disabilities. There are two phases in the project – (i) Sign language to text conversion and (ii) Audio to sign language conversion. During the process of converting sign language to text, a camera is used to capture an image of the sign language input. From these images, the characters are predicted by the neural network and the corresponding text is displayed. For the audio to sign language phase, the audio is taken from the microphone and given to the google speech API. The speech is converted into text format. The text output is compared with the data given in the database that contains Graphics Interchange Format (GIF) files and the particular text in GIF. The sign language to text and audio to sign language are both interfaced in the app. The app can perform both the case based on the requirement of the people.