Human Fear Analysis using Signal and Image Processing

Human emotion detection is an emerging field. The greater impact of emotional intelligence in day to day life than intelligent quotient has been proved by psychologists. Numerous psychological problems are coming up every day posing serious challenges. These can be solved only through proper analysis of emotions. Emotion analysis is a challenging task. Most of the time single emotion cannot be identified. Basic emotions are happy, sad, fear, anger, surprise and neutral. Fear and anger are the two dominating emotions which can cause health problems as well as mental disorder. The main focus in this paper is fear analysis using image and signal processing. In this paper, analysis of fear is made using image processing, fused facial image processing, Field Programmable Grid Array features of facial image, emotional speech processing and emotional analysis using physical parameters. Statistical feature extraction from both time and signal domain has been done. Features have also been extracted from Field Programmable Grid Array. Speech features have been extracted using Mel Frequency Cepstral Coefficients algorithm. Physical parameters which are directly related to human emotions are analysed by fuzzy analysis. Multimodal emotion analysis is done using feature level fusion. Feature level fusion is done by discrete wavelet transform and regression analysis. The features are finally classified using back propagation algorithm of conventional neural network and back propagation algorithm of convolution neural network in the domain of deep learning. Out of all emotions fear has sensitivity and specificity of 97.36% and 91.67% respectively. As against the sensitivity and specificity for only physical parameters and facial images are 58.62%, 79.41%, 81.25%, 47.62% respectively. Human fear also has been analysed from speech signal using modified Mel Frequency Cepstral Coefficients algorithm. Kaiser window works best for happiness, hamming window is good for boredom and fear, Hanning window is fit for disgust and anger, Bartlett window is good for sad emotion. Emotion detection by image fusion technique using conventional back propagation network as classifier, sensitivity and specificity are increased by 16.72% and 27.75 % respectively. Fear emotion is best classified by taking combined feature set other than single feature set like human emotional faces or physical parameters. It can also be well classified by deep neural network. The features for fear emotion can be extracted using modified Mel Frequency Cepstral Coefficients algorithm using Hamming window.