Sign Language Identification using Skeletal Point-based Spatio-Temporal Recurrent Neural Network

Sign language is a visual language that serves as the primary means of communication in the non-hearing community. It is essential to research and develop sign language translation methods to enable smooth communication between the non-hearing and the hearing communities. The non-hearing community faces several challenges that hinder their interaction with the general public, such as the lack of knowledge of sign language and the availability of interpreters. A Sign Language Converter would thus be an important tool in breaking the communication barrier between the non-hearing and hearing communities. The paper aims to build a model that converts Indian Sign Language into corresponding words. We plan on using a skeletal-point feature extraction framework to identify hand landmarks from sequences containing distinct signs and use these landmarks to build a model for recognizing hand gestures using various Long Short-Term Memory (LSTM) Networks. [1] This approach can produce an accurate result compared to the traditional approach. The dataset is made by the co-authors of this paper due to the insufficiency of pre-existing ISL datasets. The user will be monitored and using the machine learning techniques discussed above, which will perform the real-time translation to display the final result.