Global and Local Mixer for Micro-Expression Recognition

Micro-expressions (MEs) refer to spontaneous, subtle, and fleeting facial movements that usually only occur in local facial areas, which can effectively reveal genuine emotions that humans try to hide. However, micro-expression recognition (MER) is still a difficult task due to its low intensity and local movements, and existing datasets are usually small-scale with class imbalance problem. To address these problems, we propose a novel MER framework named Global and Local Mixer (GL-Mixer) based on MLP-Mixer, which can focus on local regions where MEs appear and reduce the influence of ME-irrelevant regions. Specifically, the GL-Mixer mainly consists of three parts: Global Feature Learning (GFL) module, Local Feature Learning (LFL) module, and Feature Refinement (FR) module. The GFL module is designed to learn the global ME features from the whole face region. Meanwhile, the LFL module is composed of multiple parallel local mixers, and each local mixer extracts fine-grained features in a specific facial region. Then, Feature Refinement (FR) module aims to refine the fin e-grained ME features by focusing more on the local features of the effective ME regions and suppressing the features of the irrelevant region under the guidance of the global feature. Last, the model fuses the global and fine-grained ME features for classification. In addition, to solve the problem of insufficient data and class imbalance in ME datasets, we introduce class-balanced implicit semantic data augmentation. Extensive experiments are conducted on CASMEII, SAMM, and SMIC datasets, and the experimental results demonstrate the effectiveness of our method.