Real-Time Gesture and Sentence Level Sign Language Translator

In 2019, data from the World Health Organization indicated that over 430 million individuals globally are grappling with either partial or complete hearing loss. Communication for these individuals typically involves either written expression or sign language, the latter primarily relying on hand motions. This research endeavors to craft an optimal recognition model for both Indian Sign Language and American Sign Language. The model leverages Convolutional Neural Networks (CNN), a deep learning algorithm, for gesture detection. Through skin segmentation, the regions of interest (ROI) are identified and tracked for recognizing the corresponding sign indications. The model then captures hand landmarks, storing their key points in a NumPy array. These landmarks serve as the basis for model training, with the output manifested as both English text and speech. Impressively, the CNN algorithm yields a 99.5% accuracy rate. Notably, prior studies were primarily focused on alphabet recognition, lacking emphasis on word recognition. In contrast, our model demonstrates exceptional efficiency by accurately detecting both alphabets and words, marking a significant advancement in this field.