A Novel Approach for Emotion Recognition for Pose Invariant Images Using Prototypical Networks

Facial Expression Recognition (FER) is significant in various application areas such as e-learning, surveillance, healthcare and marketing. However, FER has continued to be a challenging task for years due to various factors that needs to be considered such as arbitrary poses, occlusions, and illumination levels. Despite the surge in research using complex deep learning models to solve these difficulties, lack of good training data in terms of quality and quantity is a challenge in itself for building efficient FER models. In this work, a model called ERMLAP (Emotion Recognition using Meta-Learning for Arbitrary Poses) is proposed, that overcomes the need of having large volume of data and performs FER on images with arbitrary poses. This work uses an in-house dataset called AED-1 (Amrita Emotions Dataset-1) and a subset of the CMU Multi-PIE dataset both having five basic emotions to train and test in a few-shot regime. Two approaches are proposed in this work that use the models built by a metric based meta-learning concept called prototypical networks. In the first approach, the model was tested on ‘poses’ never seen in training and it gave 90% and 83.4% accuracy on AED-1 and CMU Multi-PIE respectively and in the second approach, the model was tested on ‘emotions’ never seen in training and it gave 92.8% and 90.1% accuracy on the two datasets.