Nonintrusive Driving Behavior Characterization From Road-Side Cameras

In this article, we demonstrate that deep learning (DL) and spatiotemporal reasoning can effectively identify driving behavior based on the videos captured by roadside cameras. The use of roadside infrastructure for such determination is twofold: 1) a global view of the vehicles and their interactions and 2) no involvement or awareness of the vehicles or their drivers, so the determination is inexpensive, easy to deploy, and entirely nonintrusive. Furthermore, our method uses DL only for object detection and tracking and builds a flexible and explainable reasoning model to identify the driving behavior. The essential advantage of this approach is that we use DL only for tasks that can be accomplished efficiently and with high accuracy (i.e., object detection and tracking), which can be done in real time. Although there are DL models for detecting complex activities (e.g., aggressive driving), they are much harder to train, require higher accuracy, and inferencing time may not satisfy real-time constraints. By using a setup with program-controlled robocars, we demonstrate that we can achieve accuracies of 98% and 99% for driving behavior characterization, and the mechanism can provide detection of 650 ms on a very dated desktop. The characterization can provide feedback to the driver (or the automated car) for improved traffic safety and roadway throughput.