Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving

Concept bottleneck models have been successfully used for explainable machine learning by encoding information within the model with a set of human-defined concepts. In the context of human-assisted or autonomous driving, explainability models can help user acceptance and understanding of decisions made by the autonomous vehicle, which can be used to rationalize and explain driver or vehicle behavior. We propose a new approach using concept bottlenecks as visual features for control command predictions and explanations of user and vehicle behavior. We learn a human-understandable concept layer that we use to explain sequential driving scenes while learning vehicle control commands. This approach can then be used to determine whether a change in a preferred gap or steering commands from a human (or autonomous vehicle) is led by an external stimulus or change in preferences. We achieve competitive performance to latent visual features while gaining interpretability within our model setup. ¹

1. Introduction

Understanding how human drivers and autonomous vehicles make decisions is essential to ensure safe and reliable operation in various real-world scenarios. Neural networks are powerful tools used for automated learning in the field of self-driving cars [2], [8], [25], [28], [30], [34], [37]. However, one significant challenge associated with deep neural networks is their nature as black-box models, which hinders the interpretability of their decision-making process. This paper proposes to address this challenge by applying concept bottleneck models for explaining driving scenarios. Concept bottleneck models incorporate vision-based human-defined concepts within a bottleneck in the model architecture [22], [31]. By encoding driving and scenario-related concepts into the decision-making process, our objective is to provide interpretable and explainable insights into the factors that influence the actions of both drivers and autonomous vehicles. Previous research has demonstrated the effectiveness of learning vehicle controls for autonomous driving [6], [24], [39], [42], [42], but the lack interpretability poses challenges to trust, safety, and regulatory compliance. The development of interpretable and explainable models has thus gained significant attention in the research community, aiming to bridge the gap between the performance and interpretability of deep learning models.