Spatio-temporal fusion for Macro- and Micro-expression Spotting in Long Video Sequences

In this paper, we aim to construct a spotting framework automatically. It is still a great challenge to spot micro-expression(ME) intervals accurately due to short duration, low intensity, and shaking. Under the uncontrolled condition, the transformation is caused by head shaking. In order to remove the global movement caused by head shaking, we propose a simple yet effective method to disentangle local movement vector from the global optical flow field by the estimation of mean optical flow in the nose region. After preprocessing, we extract the completed specific pattern(SP) of ME in each region of interest(ROI). The pattern consists of two sub-patterns: magnitude and angle. However, influenced by frame rate and different intensities of micro- and macro- expressions, we propose to use a multi-scale filter to improve the ability to spot both micro- and macro-expressions. The spotting result performed on $CAS(ME)^{2}$ and SAMM shows that our proposed method outperforms the basline method.