LSTM Based Feature Learning and CNN Based Classification for Speech Emotion Recognition

Speech emotion recognition (SER) is a challenging topic for machine learning systems since it requires the detection of a speaker's emotional state. Real-time applications like as assessing human behavior, interfacing with robots, and utilizing virtual reality all rely heavily on SER's ability to analyze emotional states. Previous work in this area has mostly concentrated on manually created features and conventional convolutional neural network techniques that extract high-level characteristics from voice spectrograms to improve recognition precision and model complexity. Instead, we provide a novel SER framework that selects key sequence segments via similarity measures among clusters in redial-based functioning networks (RBFNs). In order to extract the most distinctive characteristics of the spoken spectrogram, the STFT method is used to transform the chosen sequence into a spectrogram. We also normalize the CNN characteristics to provide reliable recognition performance. These characteristics are then used to train a deep bidirectional short- and long-term memory (BiLSTM) to distinguish an emotion's terminal state based on its temporal context. The proposed technique normalizes the CNN features before processing them, allowing it to readily identify the spatiotemporal information, and it analyzes the relevant segments rather than the complete utterance to minimize the computing complexity of the whole model. In order to enhance recognition accuracy and decrease model processing time, the proposed system is tested on industry-standard datasets as EMO-DB, IEMOCAP, and RAVDESS. The studies demonstrate the robustness and efficacy of the proposed SER model, which can achieve up to 72.25 percent accuracy over the IEMOCAP dataset, 85.5 percent accuracy over the EMO-DB dataset, and 77.0 percent accuracy over the RAVDESS dataset when contrasted to state-of-the-art SER techniques.