BISINDO Hand-Sign Detection Using Transfer Learning

This paper presents the development of a BISINDO hand sign language recognition system using transfer learning methods. The system aims to address the communication challenges faced by the deaf community in interacting with the majority of people and alleviate the feelings of isolation experienced by the deaf and mute society. Two models, SSD MobileNet v2 FPNLite 320x320 and SSD MobileNet v2 FPNLite 640x640, were trained for 20,000 steps using a custom dataset of BISINDO alphabets (A-Z). The results demonstrate the successful detection and recognition of BISINDO alphabet characters by both models. The second model, trained with SSD MobileNet v2 FPNLite 640x640 pre-trained model, exhibited slightly higher accuracy with improved mAP and AR scores. These findings highlight the potential of computer vision technology to facilitate real-time detection and recognition of BISINDO hand signs, enabling effective communication between deaf and hearing individuals. Future research can focus on refining the models, expanding the dataset, and incorporating more comprehensive sign gestures to enhance the system’s accuracy and usability. This research contributes to advancing assistive technologies for the deaf and hard-of-hearing community, fostering inclusivity and accessibility in society.

1. INTRODUCTION

Communication is an essential part of human life, and it can take various forms, including spoken language, written language, and sign language. Sign language is a visual language that uses hand gestures, facial expressions, and body movements to convey meaning. The deaf and hard-of-hearing community commonly uses sign language to communicate with each other and the hearing community.