Improving Facial Emotion Recognition Through Transfer Learning with Deep Convolutional Neural Network (DCNN) Models

Humans have a wide range of means for expressing their emotions, such as body language and facial expressions. The most well-known and thoroughly researched component of emotion recognition is facial expression analysis. Recently, the study of psychology, psychiatry, and mental health has also been increasingly interested in the problem of identifying emotion from facial expressions. However, FER still comes across numerous difficulties with automatic recognition for a variety of reasons, including feature extraction, creating models, and pre-processing sampled data. Therefore, the deep learning techniques, especially Convolutional Neural Network (CNN) models, have attracted the attention of researchers. The automatic feature extraction process is commonly employed for FER using Convolutional Neural Network (CNN). A pre-trained DCNN models is implemented by changing its top layer(s), and then the model has been modified with facial emotion dataset in order to create a highly accurate FER system. The Transfer Learning (TL) technique is used by very Deep CNN model. Facial emotion data are then used to fine-tune the model. The accuracy has been evaluated on these datasets JAFFE, CK+48, FER 2013, and KDEF. The accuracy of the Inception-v3, VGG-19, ResNet-152, ResNet-50 and DenseNet-161 models specifically improved on JAFFE by 6.45%, 10.28%,8.45%, and 9.47%, respectively. These models accuracy improved by 7.12%, 2.20%, 5.02%, and 8.91 % on KDEF, respectively. On test sets from JAFFE, the best obtained FER accuracies with DenseNet-161 is increased by 7.23%. The metrics parameters i.e., Recall, Precision, and F1 score is also calculated from confusion matrix for individual emotions to corresponding datasets.