Isolated Japanese Sign Language Recognition Based on Image Entropy Variation Rate and Score-Level Multi-Cue Fusion

The challenge in Isolated Sign Language Recognition (ISLR) research is to effectively eliminate transition frame redundancy and to integrate disparate features. To solve these two problems, We proposed an ISLR approach that employs a keyframe abstraction method based on Image Entropy Variation Rate (IEVR) and score-level fusion. First, key sign frames were extracted from sequential video frames based on IEVR. Then, four other cues were computed-hand cropping, masking, skele-tal landmarks, and optical flow-from these keyframes, for a total of five cues. Finally, these cues were used to conduct score-level fusion in different weighting strategies. The method was evalu-ated using a publicly available isolated Japanese Sign Language (JSL) dataset. The evaluation revealed the average accuracy of our IEVR-based method to be 76.71 %, which indicates that our method outperforms other keyframe abstraction methods, and the fusion result was 97.87 %, showing an improvement of 1.43 % over the best single-cue result.