Emotion Recognition Using Wavelet Synchrosqueezing Transform Integrated With Ensemble Deep Learning

This study designs a wavelet-based synchrosqueezing transform (WSST)-driven optimized ensemble deep learning (DL)-based automatic multiclass emotion recognition (AMER) system to recognize human emotions in both subject-dependent and subject-independent schemes. The 1-D electroencephalogram (EEG) signals were transformed into 2-D time–frequency representation (TFR) using the WSST method, which was given to the optimized ensemble of convolutional neural networks (ECNNs) to classify the emotions. The comparative analysis of the proposed AMER with the three TFRs [short-time Fourier transform (STFT), continuous wavelet transform (CWT), and smoothed pseudo-Wigner–Ville distribution (SPWVD] is studied. The results of both the schemes were identical, with the highest accuracy and F1-score from WSST. Moreover, WSST-driven optimized ECNN outperformed the pretrained models VGG16, ResNet50, AlexNet, and ViT-B/32 in both the schemes. A similar comparable trend is also seen in the two validation SEED and SEED_IV datasets. To summarize, this study proved that the proposed AMER system is effective, versatile, and robust, and it can be useful in emotion recognition.