Spectral and Pitch Components of CQT Spectrum for Emotion Recognition

With increasing technological advancements, man's dependence on machines is also growing. Thus, effective Speech Emotion Recognition (SER) is vital for good human-machine interaction. In this paper, we propose Constant Q Transform (CQT)-based features, namely, Constant Q Harmonic Coefficients (CQHC) and Constant Q Pitch Coefficients (CQPC) and its feature-level fusion (CQHC+CQPC) for emotion recognition, as they capture prosodic cues, which aids for SER task. Proposed features are motivated by music research (in particular, timbre and melody). In this paper, we have used Emotional Speech Database (ESD) and Toronto Emotion Speech Set (TESS) dataset, and Convolutional Neural Network (CNN) model with 10-fold cross-validation for the classification of 5 emotions, namely, anger, neutrality, happiness, sadness, and surprise. Results show that the proposed features (CQHC + CQPC) give 98.94 %, 61.77 %, and 99.56 % test accuracy whereas the traditionally used Mel Frequency Cepstral Coefficients (MFCC) give 96.63 %, 36.05 %, and 81.34 % and we obtain 98.74 %, 41.77 %, and 99.87 % with Gammatone Frequency Cepstral Coefficients (GFCC) on ESD (parallel and non-parallel) and TESS, respectively, for the SER task. The performance of CQT and Constant Q Cepstral Coefficients (CQCC) are also studied.