Arabic Sign Language Alphabet Recognition Based on HOG-PCA Using Microsoft Kinect in Complex Backgrounds

A complex background is a well-known problem in any vision-based sign language recognition system. This paper presents Arabic sign language alphabet recognition system in complex backgrounds using Microsoft Kinect. The proposed system passes by three phases. The first phase which is the signer segmentation process, focuses on the isolation of the active signer rapidly from the background. The system assumes that the active signer is the closest person to the Kinect sensor, so that the system isolates this person from any other persons or any skin-like object that may exist in the scene. After that, hand segmentation is achieved using RGB-Ratio color model. Histogram of Oriented Gradients (HOG) is extracted from the image then Principal Component Analysis (PCA) is applied on HOG so that HOG-PCA is used to train a support vector machine (SVM) classifier. The system is able to recognize the 30 Arabic alphabets with an accuracy of 99.2%. The proposed online system has the ability to recognize Arabic alphabet almost correctly in a reasonable response time.