ECCNet: An Ensemble of Compact Convolution Neural Network for Pain Severity Assessment from Face images

Pain is an indication of physical discomfort, and its assessment is crucial for the medical diagnosis of the patient. Currently, the self-report are known to be the gold standard for pain assessment. However, it is highly subjective and prone to human errors. Therefore, in this paper facial expression based fully automated pain severity assessment system is proposed. Although significant work is done within the pain research area from facial expressions, the volume of the research has been conducted for either binary (pain/no pain) or four class classification problems. Moreover, previous approaches utilized a single neural network with a large number of trainable parameters. A single deep neural network might not achieve optimum performance due to instability. Hence, in this study Ensemble of Compact Convolutional Neural Networks (ECCNet) has been proposed for assessing various pain intensities from the facial images. The proposed system represents an effective way of boosting the performance of the pain classifier by combining various heterogeneous compact CNNs into an ensemble. The proposed ECCNet system utilizes multiple CNN topologies along with different configuration settings. Here, we have suggested that learning of multiple heterogeneous compact CNNs results in more generalized pain classifiers than learning from independent baselearners. Given this, the proposed system utilizes three compact CNNs (i.e. variant of VGG, MobileNet, and GoogleNet) and integrated their predictions using the average ensemble rule. Moreover, various augmentation techniques are used to improve the generalization capability of each network. The proposed system has been evaluated extensively on the UNBC-McMaster shoulder pain dataset for five-level of pain intensities. The experimental results demonstrate the significance of the proposed ensemble method, which achieved 91.41 % of F1-score for pain severity assessment.