An Unsupervised Learning Approach to Handle Movement Epenthesis in Continuous Sign Language Recognition

In this paper, the problem of movement epenthesis in continuous sign language sentences is considered. Movement epenthesis caused due to unwanted but unavoidable hand movement in between two sign gestures in continuous signing has emerged as one of the most challenging problems in automatic sign language recognition. To handle this problem, a novel method based on unsupervised learning approach has been proposed in this paper to separate out video frames corresponding to meaningful sign gestures from meaningless movement epenthesis segments in a continuous signing gesture video clip. Our proposed method is based on K-means clustering of the norm values of the absolute difference between current frames and the reference frame to detect the movement epenthesis frame and then classify the frames of the video as movement epenthesis frames and sign frames. Exhaustive experimentation on the publicly available standard ChaLearn LAP ConGD gesture video dataset was carried out to test our algorithm. Experimental results demonstrate that the proposed method is good enough to detect and separate out the movement epenthesis frames in sentence level sign language videos. Our proposed approach performs the movement epenthesis detection task accurately in 91% of the videos of the ChaLearn LAP ConGD dataset.