Pain Intensity Recognition from Masked Facial Expressions using Swin-Transformer

Previous studies recognize pain expressions based on the entire face, for example, Prkachin and Solomon Pain intensity (PSPI). However, the patients face is often masked by instruments in an intensive care unit (ICU), such as respirator, and gauzes, just name a few, which causes the agent cannot measure the pain intensity using PSPI directly. To tackle this problem, we explore the pain intensity recognition from masked face. First, we conducted four levels of pain measurement experiments with four types of masked face using Swin-Transformer. Experiment results show that the accuracy of pain measurement is more than 90%, even masking a large part of facial Action Units (AU) related to the pain on the UNBC-McMaster dataset. Furthermore, we pre-train the pain recognition model with masked face dataset, such that the model can capture facial features contributed to the pain. Results on binary and four-level pain intensity measurement tasks show our model outperforms recent state-of-the-art performance, achieving 97.38% and 95.25% accuracy, respectively.