Speech Emotion Recognition Using Combined Mel Spectrograms with 2D CNN Models

Speech emotion recognition is a task in machine learning to classify audio signals according to the emotions portrayed by the speaker. Spectrograms allow audio classification task to be treated as a computer vision task and use similar methods and models. Many Convolutional Neural Network (CNN) models have been created and continuously refined for computer vision tasks. This paper compares two well-established CNN models, namely VGG16 and Xception. Additionally, a method of feature extraction is proposed, which offers additional benefits when utilizing RGB channels in 2D CNN models. The proposed method provides a boost classification accuracy from 80.4% to 82.7% using VGG16 and from 81.5% to 84.1% when using Xception CNN and by RGB features of preprocessed signals.