Research on Lip Recognition Algorithm Based on Optimized MobileNet

Lip recognition is the process by which a computer learns the changes in dynamic lip image sequence of the speaker's lips to recognize the content of the language spoken by the speaker. Lip recognition technology is widely adopted in the fields of national public safety protection, medical correction, etc. Most of the current research uses deep learning for lip recognition, however, in order to achieve high recognition rates, the network models for lip recognition are getting larger and larger, which are difficult to deploy to mobile devices. Therefore, in this paper, we optimize and improve the traditional MobileNet network to address the above problems, and propose a more lightweight FD-MobileNet network, and use the combination of FD-MobileNet network and GRU network to recognize the two-dimensional image features and temporal features of lips. FD-MobileNet can extract the two-dimensional features of images, and GRU network can learn the action changes between sequences. Based on the advantages of these two networks, we propose a combination of FD-MobileNet and GRU for lip recognition. We analyze the performance of the model by recording the change curve of loss function, the performance of common models in self-made datasets and the lip shape difference of words. The result shows that FD-MobileNet + GRU network model possesses the best effect for lip recognition. (Abstract)