Facial Expression Recognition System with Local Binary Features of Neural Network

The use of accurate and faster algorithms for real-time automatic facial expression detection is a challenge. Facial expression recognition (FER) has three important steps: face detection, facial feature extraction, and emotion classification based on the extracted features. Face detection comes under the pre- processing stage, where the face of the object can be recovered for easy extraction of the features. But, the most important part of FER is carefully selecting the features responsible for detecting the expressions and their extraction from the face. Nowadays, mostly advanced neural networks are used for the classification of expressions based on the extracted features, which are also very complex and time-consuming. Further, the decision trees and the neural networks increase the recognition rate of facial expressions into many folds. The requirement of the number of extracted features also increases to achieve better accuracy. Hence, the storage requirement also increases along with an increase in the complexity and cost of the complete system. This work tries to reduce the number of features by focusing only on local binary features, reducing the storage, complexity, and overall cost of the complete FER system. An improved pre-processing stage is used in the proposed work. Further, it has an important local feature- based learning step that includes the extraction of local binary features for expression classification. These feature vectors are concatenated and used in Shallow neural networks with minimum complications and a smaller number of layers for optimizing the expression recognition process along with gentle boost decision trees. This method of local binary features-based neural network (LBF-NN) is applied to three different popular databases, and it yielded favorable results of more than 93% accuracy, even in comparison with various complex and advanced algorithms.