Speech Emotion Recognition Using Deep Learning on audio recordings

Speech emotion recognition plays a prominent role in human-centred computing. An intelligent agent should be able to extract the context of the speech of a person including the emotion underlying in that speech. This will enable the agent to provide sensible feedback. However, automatic recognition of emotions from the speech is a challenging task due to the complexity of emotional expressions. It is still unclear that, which features of a human speech are robust enough to distinguish emotions and research in this field continues. In most of the machine learning applications, the features representing the entities in concern should be identified by an expert and then needed to be extracted. The performance of most of the Machine Learning algorithm depends on how accurately the features are identified and extracted. Deep learning has emerged as an advancement of traditional machine learning. Deep learning algorithms are capable to learn high-level features from raw data itself. Due to these improvements, recent approaches to emotion recognition utilize various deep learning models. These models were built using advanced representations or pre-processed speech recordings such as spectrogram and time-frequency representation of speech. This work proposes an end-to-end deep learning approach which applies deep neural network on a raw audio recording of speech directly to learn high-level representations from the audio waveform. The performance of the proposed model was assessed on USC-IEMOCAP and EmoDB datasets. The CNN model with nine weight layers obtained an overall accuracy of 68.6% for IEMOCAP over four emotions and 85.62% for EmoDB over seven emotions. Obtained results show that the performance of the proposed model is comparably better than the performance achieved by traditional machine learning approaches as well as deep learning on spectrogram of audio recordings in the same datasets.