Automatic Evaluation of Facial Paralysis with Transfer Learning and Improved ResNet34 Neural Network

Facial paralysis is a relatively frequent disease characterized by facial muscle movement disorders. Clinically, the patient's facial paralysis is usually assessed by the clinician according to assessment scales. The most popular scale for assessing facial paralysis in recent years is the Sunnybrook Facial Grading System (SFGS). However, the scale's assessment results are highly subjective and inefficient. Using artificial intelligence technology for the auxiliary diagnosis of facial paralysis assessment can improve the accuracy and objectivity of diagnosis. But the existing method or model based on computer vision for facial paralysis auxiliary diagnosis assessment mainly suffers from insufficient datasets and inadequate extraction of useful facial feature information. Therefore, this paper proposes an objective method for facial paralysis assessment based on transfer learning and an improved ResNet34 network model. This assessment method is based on SFGS, which assesses the degree of facial paralysis separately for the patient's eyes, nasolabial folds, and mouth when the face rests. Firstly, the keyframe images are extracted from the patients' captured facial videos, and the data's validity is increased by preprocessing with geometric normalization and grayscale normalization. Then, for the strengthened feature extraction problem, an improved resnet34 network (named ResNet34-SP) is designed with SoftPool as the pooling layer of the network, which retains more feature information. Finally, the Extended Cohn-Kanade (CK+) database was used for transfer learning pre-training to improve the problem of insufficient datasets and realize the automatic SFGS static score assessment of facial paralysis. The accuracy rates for the eyes, nasolabial folds, and mouth were 97%, 87%, and 88%, respectively, confirming the method's effectiveness.