Applying Image Classification Model to Spectrograms for Speech Emotion Recognition

Speech Emotion Recognition (SER) aims to enable machines to comprehend the subjective emotions of individuals by analyzing their vocal expressions. The challenge of SER lies in extracting and exploiting the comprehensive sound information effectively. In this paper, we propose an end-to-end SER system that utilizes multiplexed acoustic informations, including visual representations of spectrograms and Mel-frequency cepstral coefficients (MFCC). Specifically, to capture salient sound signal features effectively, we employ a pre-trained EfficientNet to analysis the spectrogram, and a Bidirectional Long Short-Term Memory (Bi-LSTM) network to process the MFCC. Then the extracted features are combined using attention modules. Results of experiments conducted on the RAVDESS database using a five-fold cross-validation strategy demonstrate the competitive performance of the proposed SER system to state-of-the-art methods.