Deep learning approach for response assessment to low intensity emotional stimuli

This study provides a framework for objectively predicting emotional states based on a set of multimodal physiological data. It underscores the significance of establishing robust systems for emotional state prediction by integrating technologies such as multimodal physiological data analysis with advanced deep learning algorithms. This provides experts with a tool that can assist in various scenarios, including health, psychological assessment, and recreation, where subject evaluation is required. Employing objective assessment methodologies, the research aims to offer a more accurate evaluation of mental states compared to subjective methods. The study utilizes a dataset comprising multimodal physiological data collected on 30 subjects during emotion-evoking stimuli sessions. In this study, the electrocardiographic data, the respiratory data, the photoplethysmographic data, and the electrodermal activity data are considered for the analysis. Key steps include data preprocessing, filtering and normalization, and the development of a deep learning framework combining Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. The paper presents results indicating the model’s proficiency in classifying emotional responses across different categories, although challenges remain in certain classifications. The final model achieved promising results, with precision reaching 69% and sensitivity reaching 81% in differentiating between binary emotional responses. However, the model faced challenges in distinguishing specific emotional states in ternary and quaternary classifications, with an accuracy value hovering around 50%. Further research could explore the integration of additional modalities and transfer learning techniques to enhance model accuracy and generalization ability in diverse scenario.