Driver’s Drowsiness Classifier using a Single-Camera Robust to Mask-wearing Situations using an Eyelid, Lower-Face Contour, and Chest Movement Feature Vector GRU-based Model

Drowsy drivers cause many deadly crashes. As a result, researchers focus on using driver drowsiness classifiers to predict this condition in advance. However, they only consider constraint situations. Under highly unrestricted scenarios, this categorization remains extremely difficult. For example, several studies consider the driver’s mouth closure crucial for detecting drowsiness. However, the mouth closure cannot be seen when the driver wears a mask, which is a potential failure for these classifiers. Moreover, these works do not make experiments under unconstrained situations as environments with considerable light variation or a driver with eyeglasses reflections. As a result, this paper proposes a video-based novel pipeline that employs new parameters, computer vision and deep-learning techniques to identify drowsiness in drivers under unconstrained situations. First, we alter the Lab color space of the frame to ease strong light changes. Then, we achieve a robust recognition of the face, eyes and body-joints landmarks using dense landmark detection that includes optical flow estimation methods for 3D eyelid and facial expression movement tracking and an online optimization framework to build the association of cross-frame poses. After this, we consider three important landmarks: eyes, lower-face contour, and chest. We performed several pre-processing and combinations using these landmarks to compare the efficiency of three alternative feature vectors. Finally, we fuse spatiotemporal features using a Gated Recurrent Units (GRU) model. Results over a dataset with highly unconstrained driving conditions demonstrate that our method outperforms classifying the driver’s drowsiness correctly in various challenging situations, all under mask-wearing scenarios.