A Robust Pose & Illumination Invariant Emotion Recognition from Facial Images using Deep Learning for Human-Machine Interface

Human facial emotion recognition has been studied for decades as facial expression confers added impact in recognizing human emotions and improves communication. Recently, emotion recognition through facial images has become an active research area with the evolution of deep learning in computer vision. This paper presents work on emotion recognition through facial images using layer based Convolutional Neural Networks (CNN). The proposed method for emotion recognition is named as DPIIER (Deep learning Pose Illumination Invariant Emotion Recognition). In this work, we have used 15 layers for constructing CNN that learns from CMU Multi-PIE facial expression dataset images, to recognize and classify five basic emotions: Neutral, Anger, Happy, Surprise and Disgust. The uniqueness of DPIIER lies in the size of input images with different pose and illumination variations. We have used 32×32 small size images for training and testing of emotion recognition and the developed algorithm is pose and illumination invariant. To test the robustness of the algorithm, III-fold cross validation has been performed and the average accuracy achieved is 96.55% by using CNN based algorithm on Graphical Processing Unit (GPU). The processing time for training and testing are 4 min and 110–200 m sec respectively. The cross dataset validation is performed using different datasets JAFFE, CK+ and KDEF.