Gender Differentiated Convolutional Neural Networks for Speech Emotion Recognition

This paper proposes a two-stage gender-differentiated system for Speech Emotion Recognition using Mel-frequency Cepstral Coefficients and Convolutional Neural Networks. Acoustical variances between male and female speakers pose a problem and it is established that gender-dependent emotion recognizers perform better than gender-independent ones. The provided solution can recognize seven emotions (anger, disgust, fear, happiness, sadness, surprise, and neutral state). Data augmentation is used to compensate for the lack of quality data, with the raw speech samples derived from four datasets, namely: RAVDESS, CREMA-D, SAVEE, and TESS. The system is composed of two stages: 1) gender classification and; 2) emotion classification. The output of the gender classifier in the first stage determines the gender-specific classifier for the second stage. The experimental evaluation displays the performance in terms of the correct emotion recognition rate of the proposed SER model. The results demonstrate that a gender-differentiated system significantly improves performance. The obtained results also show that using Global Average Pooling instead of a fully-connected network at the end of the CNN classifier further improves the performance. Future implementations of this proposed system may allow effective human-computer intelligent interaction.