Spatial Ensemble Distillation Learning for Large-Scale Real-Time Crash Prediction

Large-scale real-time crash prediction is critical to traffic operation and management, but very challenging, even for machine learning models because the observation data are not independent and identically distributed (non-IID) due to its spatial heterogeneity. Data discontinuity, computational inefficiency, and higher false alarm rates are some other challenges in real-world and real-time crash prediction. To address these issues, we propose a generic framework, which blends spatial ensemble learning and knowledge distillation. Spatial ensemble learning leverages individual segment-level modeling to alleviate the non-IID issue for more accurate crash prediction, while knowledge distillation reduces the model size and improves false alarm rate. We justify the effectiveness of the proposed method using real-world and real-time scenario and comparing the results with the up-to-date benchmark models. Our method successfully improves sensitivity and false alarm rate to 90.35% and 24.21%. With this more accurate prediction model, we analyze the prediction variability across this region. We found that: 1) across segments, false alarm rate exhibits more heterogenous pattern than sensitivity; 2) rear-end crashes are more accurately predicted compared to sideswipe/angle crashes; 3) urban segments show better prediction performance compared to rural segments. 4) developing models with desired accuracy requires special attention in higher traffic fluctuation segments. These observations are very useful to develop more accurate prediction model and traffic safety decision making. To the best of our knowledge, this is one of the pioneering studies to integrate spatial ensemble learning and knowledge distillation to predict large-scale real-time crashes and apply it to analyze crash prediction variability.