Emotion Classification of User Face Image in Music Recommendation System

A person could listen to different music when sad or happy. Listening to favorite music stimulates the brain to release dopamine hormone to the corpus striatum, which manages human feelings such as addiction, satisfaction, and motivation. So human emotion could be an opportunity to enhance the music recommendation system. Gilda et al. (2017) and Krupa et al. (2020) used Convolutional Neural Network (CNN) to classify emotion from face image with FER2013 dataset. The performance of Krupa’s CNN is not good yet and unequally to each emotion. While Gilda’s CNN performance is good, the convolution layers quite a lot (9 layers) with big filters (256). In a recommender system, the incomplete cold start could happen if a user has a lack of rating. So the system needs extra information to give better recommendations. Thongsuwan et al. (2021), designed ConvXGB. Two convolution layers are used for extracting input and XGBoost does the learning task. The model has better performance than CNN in the DrivFace dataset. So ConvXGB was implemented in this research for emotion classification. Emotion is mapped with the mood attribute in the music dataset (Emotify). The mood is the user’s additional information for solving incomplete cold start. The system recommends a list of music that is liked in a particular mood by a user and sorted in descending. The performance of ConvXGB in emotion classification with 128 filters and max-pooling is better than the two CNN in the oversampled dataset. ConvXGB gained 78.64% accuracy on 4 emotions and 80.99% on 7 emotions. Evaluation for incomplete cold start is using 10% to 90% of each user’s ratings in a mood as train data. The result is incomplete cold start that can be solved with mood information using 10% of each user’s ratings. The system tends to have better performance with the increment of user rating data.