Facial Attention based Convolutional Neural Network for 2D+3D Facial Expression Recognition

Discriminative facial parts are essential for facial expression recognition (FER) tasks because of small inter-class differences and large intra-class variations in expression images. Existing methods localize discriminative regions with the aid of extra facial landmarks, such as action units (AU). However, it consumes a lot of manpower in manually labeling. To address this problem, in this paper, we propose an advanced facial attention based convolutional neural network (FA-CNN) for 2D+3D FER. The main contribution of FA-CNN is the facial attention mechanism, which enables the network to localize the discriminative regions automatically from multi-modality expression images without dense landmark annotations. Experimental results conducted on BU-3DFE demonstrate that FA-CNN achieves state-of-the-art performance comparing with the existing 2D+3D FER techniques, and the discriminative facial parts estimated by the facial attention mechanism are highly interpretable and consistent with human perception.