A System For Drivers’ Cognitive Load Estimation Based On Deep Convolutional Neural Networks and Facial Feature Analysis

The driver's cognitive load estimation on the road is a critical factor for safety measurement that can reduce the overall number of road accidents. Factors such as distraction from driving, being "lost in thought," conversation, cell phone use, and drowsiness, can be used to compute a driver's attention level on driving. To minimize the number of fatal and critical circumstances on the road, this work investigates a decision tree-based deep-learning algorithm focusing on the estimation of the driver's cognitive load. We construct and evaluate the performance of multimodal deep learning approaches, which combine two separately trained ResNet50 convolutional neural networks on data acquired from the driver's side and front face images. Further, we compare two different approaches for driver drowsiness level estimation using different computer vision algorithms. The first approach is implemented via the blinking ratio method using 68-landmarks of the face by calculating the blinking and yawning ratio. In contrast, the second one uses the contour area approach for drowsiness identification using morphological operations. As differences between driver behavioral classes are more distinct from side camera images than from the frontal image, the former has been chosen to be used for deep learning classification, and eventually, the accuracy of 92% has been achieved. On the other hand, the frontal image was successfully used to robustly detect the driver's drowsiness level by using computer vision techniques on facial landmarks. As a result, the decision tree was constructed to estimate the driver's driving safety level based on deep learning side posture classification and blinking ratio methods.