Real-Time Sign Language to Text and Speech Translation and Hand Gesture Recognition using the LSTM Model

Communication is very important between People to people. People who are “Specially Abled,” with speech or hearing impairments, “Mute” or “Deaf” individuals, respectively, constantly rely on visual communication. Due to a lack of knowledge in sign language, people without visual or hearing impairments occasionally encounter problems and cannot interact with those seeking their help. They accept sign language well and make use of it to communicate. Building a system that can translate gestures into text and voice is necessary to enable two-way communication between persons with disabilities and the broader population. Translating these sign languages into text and voice can be highly beneficial since it will link the deaf/mute and the rest of society by enabling communication between the two groups. This research paper proposes a method for recording hand movements (sign language). It makes use of the Mediapipe platform, which successfully recognizes the hand and extracts the relevant information to feed into the model. After the model recognizes the sign language, words and alphabets are merged to form a sentence, which is then converted to speech. With LSTM, the suggested framework has demonstrated effective accuracy. According to a survey, CNN and LSTM produce results that are superior in quality and accuracy.