Smart Mirror for Personalized Content Delivery Through Facial Emotion Recognition

As technologies emerge to integrate many devices and appliances in a smart home environment, smart mirrors have a potential of being used for personalized content delivery to smart home users. In this study, we have developed a smart mirror using internet of things and deep learning technologies to provide users daily news updates and current events based on the user emotion. In addition, the smart mirror will also display current weather information based on the location, calendar information, to-do-list, traffic updates, currency rates, reminders, etc. The proposed smart mirror is powered by a Raspberry Pi and provides facial emotion classification using a FER CNN deep learning model. The deep learning emotion classification model was implemented on an embedded device with minimum computational resources and was able to classify emotions to three emotion labels: happy, sad and neutral with training and validation accuracies of 84.17% and 81.69% respectively. Furthermore, the proposed system proved that it could deliver personalized content based on user emotion, as it recorded an experimental accuracy of 80% when it was tested with 60 test-cases from 10 different pre-registered users of the system.