Micro-Expression Recognition Based on 2D-3D CNN

As a special facial expression, micro expressions have subtle muscle movements, which occur at a very short duration. There are many promising applications of micro expression recognition, such as criminal investigation and public security. Due to the facial motion features of micro expression are difficult to capture, it is difficult to perceive and interpret the micro expression. In this paper, a new 2D-3D CNN for micro expression recognition is proposed, which consist of Net-A and Net-B.Net-A is used to extract spatio-temporal features, and consists of multi-scale 1D temporal convolution and 2D spatial convolution. Multi-scale 1D temporal convolution is used to extract the dynamic information in micro expression video, and 2D spatial convolution is used to extract the spatial information of single frame image. Then the temporal and spatial features are fused multiple times and get the short-term spatio-temporal features. The Net-B, consists of residual blocks, is used to process differential information image and obtains the long-term spatio-temporal feature. Micro expression video contains too much temporal redundancy information, so a differential information image is proposed. Redundant features can be removed by using Net-B to extract features from differential information images. Then, the differential information feature map is fused with the features extracted by Net-A to enhance the motion features in the micro expression video. The effectiveness of the proposed 2D-3D CNN is evaluated on two open datasets: CASME and SAMM. The experimental results show that our method can achieve a better performance than the existing state-of-art methods.