Automated translation for Yemeni's Sign Language to Text UsingTransfer Learning-based Convolutional Neural Networks

Self-expression and understanding people language are among the most important things to be considered. Deaf and dumb is a group of that they facing difficulty to express themselves and communicate with others. This group of people is trying to communicate with others using “sign language”. This study is design to enhance the communication channel between these people with their society using technology. A prototype system was design to translate the Yemeni sign language into text. Deep learning algorithms included in the system using convolutional neural network (CNN) with various transfer Learning models. System evaluation is used torch and tensorflow libraries as training and testing dataset of Yemeni sign language. Accuracy comparison results obtained among various models included in this study such as Visual Geometry Group (VGG16), Residential Energy Services Network (ResNet), Google Network (GoogleNet), and Densely Connected Convolutional Network (DenseNet). We found that the accuracy results obtained for each model were (ConveNet = 98.66%), (Sequential CNN= 98.34%), (GoogleNet = 98.36%), (Vgg16 = 90,46%), (DenseNet = 99.65%), and the best result was (ResNet152 = 99.78%). This study showed the ability of technology to enhance the communication methods between deaf and their society with a suitable translation accuracy.