Real-time Mexican Sign Language recognition

The Mexican Sign Language (MSL) has its own structure and grammar different from the Spanish spoken in Mexico. The MSL is composed of more than 1000 signs used by the deaf community of this country. Few hearing people know and use MSL, therefore there is a communication barrier between hearing and deaf people. In order to overcome this issue, we propose a non-invasive sign detection application, so that the interlocutor interacts with the system in a natural way interpreting a set of signs of the MSL using a bimodal cognitive vision system. Our application uses a Kinect sensor to obtain the gestures, geometric moments and Hidden Markov models to process it. From experiments performed with 10 people, it was concluded that our application achieved an average sensitivity of 86% and an average specificity of 80%.

I. Introduction

Mexican Sign Language (MSL) consists of a series of articulated gestures signs with a linguistic function that is part of the linguistic heritage of the community and is as complex in grammar and vocabulary as any oral language. The MSL is officially recognized as a national language used by the deaf community in Mexico, and is part of the linguistic heritage of the Mexican nation.