Hands Sign Language and Gesture Recognition Using CNN

One of the non-verbal communication techniques utilised in sign language is the hand gesture. People with speech or hearing impairments use it most frequently to converse with one another and with nondisabled persons. Several makers around the world have created various sign language systems, but they are neither adaptable nor cost-effective for end users. So, the “Hand sign and gesture recognition system software” presented in this proposal offers a system prototype that can automatically understand sign language to aid deaf and dumb individuals in communicating with each other and other people more efficiently. This system can also be used in platforms like “YouTube” videos where there is currently no feature for auto-text generation on the basic of gestures and sign languages. Research on gesture recognition is still in its early stages. Hand gestures play a crucial role in nonverbal communication and are essential to daily living. The software aims to present a real-time system for hand gesture and sign recognition on the basis of detection of some shape-based features like orientation, Center of Mass centroid, fingers status, and thumb in positions of raised or folded fingers of hand, although these feature extraction part will be totally resolved by Convolutional Neural Network (CNN). For the purpose user has used ISL(Indian Sign Language) for case study. The back projection histogram approach was employed in this model to set the image’s histogram. For training and testing, used CNNs and as a result, the test accuracy was 99.89%. This model’s independence from external hardware or devices is one of its benefits.