Complex Feature Information Enhanced Speech Emotion Recognition

Speech emotion recognition (SER) helps machines behave and communicate in a more human-like way, thus has attracted extensive attention from industry and academia. Traditional SER algorithms usually discard the phase information in the speech spectrum. However, phase information plays an important role in many speech-related tasks according to recent speech-processing researches. In the SER area, the pre-trained Wav2Vec 2.0 model reached SOTA performance, still, no attention was paid to the complex information in speech. In this paper, we propose a novel framework utilizing both the real and the imaginary parts of the complex feature obtained by speech short-time Fourier transform to improve the SER performance. Specifically, we construct a deep complex neural network (DCNN) to handle the information in the speech complex feature, and then the DCNN is fused with the Wav2Vec 2.0 model to enhance the SER performance. Experiment results show that the proposed framework benefits from the compensation of DCNN and obtains nearly 1 percent performance improvement over model Wav2Vec 2.0 in terms of accuracy and F1.