Two-way Communication: An Integrated System for American Sign Language Recognition and Speech-to-Text Translation

Sign language serves as a means of communication for individuals who are deaf or have hearing impairments, enabling them to interact within their own community and with people from other backgrounds. It is a visual language that expresses ideas using body language, hand gestures, and facial expressions to convey meaning. There are over 300 different sign languages used throughout the world, each with its own distinct characteristics and regional variations. American Sign Language (ASL) has been used in this project. Many individuals do not know sign language. Hence, real-time sign language detection is one of the potential applications for overcoming the communication barrier between such people and the deaf community. Computer vision-based detection of sign language ranges from recognition of sign gesture to text and speech generation. In this paper, we attempt to create a system that establishes two-way communication between someone who signs using ASL, a deaf or hard-of-hearing individual (the signer) and someone who cannot sign (the non-signer) using computer vision. The system consists of gesture recognition using computer vision, text-to-speech conversion, and speech-to-text conversion. The system is an application of human-computer interaction which consists of gesture recognition.