A CNN Ensemble Approach to Provide Non-combat and Support Roles Training Deaf and Dumb Individuals using Hand Signs

Appointing deaf and dumb individuals in military services is very challenging, as it is very difficult process to provide them training. Machine learning model is a useful tool for providing effective non-combat and support roles training to deaf and dumb individuals through hand signs interpreters. We have developed a CNN Ensemble approach to recognize and classify various hand signs used in during training sessions. This can help interpreters to quickly and accurately interpret the commands given by military trainers, making training more efficient and effective. We have made a comprehensive dataset for this research purpose. We have designed this dataset for 35 common commands used by military trainers during training. For each of command class we have taken 160 images. So our dataset consists of total 5600 images. To design our final model, we have proposed two Convolutional Neural Networks model. Then we have imported VGG19 using TensorFlow and Keras libraries. We have applied these three models on our dataset. To get better performance we have also applied Voting Ensemble model on these three models. We have also combined these three models to make an Average CNN model and applied several classifier models such as SVM, RF, KNN and DT on that model. As we have noticed, Voting Ensemble model performed better than all these models with highest accuracy 94.20%. We believe that our proposed models will bring a great impact in defence sector.