Research on End-to-End Continuous Sign Language Sentence Recognition Based on Transformer

In 2021, the World Health Organization estimates that there are approximately 70 million deaf mutes in the world. At present, the method that facilitates the communication between normal people and deaf mutes is still not widely available. In the era of rapid development in the field of artificial intelligence, sign language recognition technology based on deep learning and mining human visual and cognitive laws has become an effective tool. In this paper, a Transformer based end-to-end continuous sign language sentence recognition model (TrCLR) is established. The CLIP4Clip video retrieval method is used for feature extraction, and the overall model framework uses an end-to-end Transformer structure. The sign language data set (CSL data set) is used as the data of this experiment. Nine sign language recognition models are used for experimental comparison on this data set. The experimental results show that the accuracy of TrCLR reaches 96.3%, which is 13.9% improvement over the best results of other models. Our model promotes the communication between normal people and deaf-mute people, and contributes to the establishment of a barrier free society.