Eye Importance in Facial Expression Recognition

The human face is a powerful tool for nonverbal communication, capable of conveying a wide range of emotions. Previous research has shown that facial expressions contribute significantly to interpersonal communication, indicating that 55% of information is conveyed through facial expression alone. Despite advancements, Facial Expression Recognition (FER) technology faces challenges, particularly in scenarios involving occlusions. Instances during the COVID-19 pandemic and in Virtual Reality (VR) environments highlight these challenges, where mask usage and head-mounted displays obstruct facial features critical for accurate recognition. This paper aims to investigate the importance of the eyes in facial expression recognition by using four models: ResNet-18, VGG-19, EfficientNet-B1, and an Ensemble model. Utilising the FERPlus dataset, scenarios with and without occlusion were examined. In scenarios without occlusion, ResNet-18 emerged as the top-performing model, achieving 86.1% accuracy. However, when occluded by goggles, the Ensemble model demonstrated superior performance with 82.8% accuracy. Furthermore, in the presence of mask occlusion, EfficientNet-B1 exhibited the most robust performance, achieving an accuracy of 71.2%. Despite challenges, the results of this paper reaffirm the enduring importance of the eyes in facial expression recognition, emphasizing their pivotal role in conveying emotions even amidst technological obstacles.