Hand Sign & Gesture Recognition System

Hand gestures serve as a form of non-verbal communication utilized in sign language, primarily employed by individuals who are deaf or have hearing impairments to communicate with those who do not share the same condition. While numerous sign language generators exist, none of them offer customization options or cost-effectiveness for end users. This proposed work outlines a prototype system named the Hand Sign and Gesture Recognition System Software. The system aims to automatically interpret sign language, facilitating smoother communication between individuals who are hearing and those who are deaf. Particularly, the system can be implemented on popular video-sharing platforms like YouTube, which currently lack an auto-text creation feature for gestures and sign language. The system utilizes a real-time hand gesture recognition mechanism that employs shape-based features such as orientation, centroid of the center of mass, finger state, and thumb position relative to extended or folded fingers to identify gestures. A convolutional neural network (CNN) supervises the entire feature extraction process. Each frame of the video is captured, allowing the identification and isolation of hands for processing by the CNN. The system achieved a remarkable test accuracy of 99.89% by employing CNNs for both training and testing. An advantage of this approach is its independence from external hardware or devices, as it operates solely through the software implementation.