Interpretable Prediction of Pedestrian Crossing Intention: Fusion of Human Skeletal Information in Natural Driving Scenarios

Given that the standardization of automated driving scenario testing is currently underway in various regions, this paper focused on the pedestrian crossing case outlined in a recent proposed international standard for terminology definitions. This study extracted human skeletal information from the Joint Attention in Autonomous Driving (JAAD) dataset, serving as a straightforward means to extract posture features. After defining a novel array of static and dynamic skeletal features, we conducted a comparative analysis of four models across two scenarios: pedestrians walking individually or in groups. To gain a more comprehensive understanding of the underlying mechanisms influencing pedestrian crossing decisions, we employed SHapley Additive exPlanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) to provide precise insights into both global and local predictions. The results show that heightened willingness to urgently cross the street is correlated with more noticeable knee flexion and leg alternation, a narrower shoulder in sight, and larger strides. Conversely, frequent body rotation may suggest a temporary reluctance to cross. Additionally, there is an indication that pedestrian crossing intention can be influenced by group size, vehicle movement and contextual features. Finally, practical suggestions based on our results are provided for automated driving scenario testing.