MM-TrafficRisk: A Video-based Fleet Management Application for Traffic Risk Prediction, Prevention, and Querying

This paper introduces MM-TrafficRisk, an innovative fleet management application that harnesses dashcam video data, environmental data, and physiological data to forecast, mitigate, and investigate traffic-risk events while uncovering traffic-risk patterns. To provide a comprehensive overview of this application, we outline its system architecture, encompassing a database, ETL processes, UI/UX components, and a fine-grained text-video search engine. Additionally, we present a groundbreaking two-stage near-miss accident prediction model designed to identify near-miss incidents within dashcam video databases, and the text-to-video search engine, facilitating rapid searches for traffic-risk events based on textual queries. These models and search engines are rigorously evaluated in a controlled laboratory environment to showcase their advantages. Moreover, we highlight several essential functions of the MM-TrafficRisk application through snapshots, emphasizing the collaborative efforts between a government agency and industrial companies to develop and deploy this application in practical settings. We also delve into our future endeavors, focusing on multi-modal deep learning event prediction and the adaptability of our application to Edge AI environments.