Sign Language Detection Using Mediapipe and Long-Short Term Memory Network

Although humans generally use verbal communication methods to interact with each other, some have limitations to the use of this technique due to hearing or speaking impairment. This category of people is known as the deaf, and they use sign language to communicate. BISINDO is one of the sign languages used in Indonesia, but due to its unpopularity, communication between the deaf and ordinary people becomes difficult. Therefore, this study developed a deep learning model to identify sign language movements using the Long Short-Term Memory Network architecture. The model was designed in three scenarios produced with the best performance identified in the first scenario and analyzed with several tests. The result showed that the first model had training and validation accuracies of 98.352% and 97.273%. In testing using the data-test, the model has an accuracy of 99% and a ROC-AUC score of 99.995%. Finally, in the actual data testing, the model has an accuracy of 89% and a ROC-AUC Score of 98.390%.