Autonomous Intersection Management with Heterogeneous Vehicles: A Multi-Agent Reinforcement Learning Approach

While autonomous intersection management (AIM) emerges to facilitate signal-free scheduling for connected and autonomous vehicles (CAVs), several challenges arise for planning secure and swift trajectories. Existing works mainly focus on addressing the challenge of multi-CAV interaction complexity. In this context, multi-agent reinforcement learning-based (MARL) methods exhibit higher scalability and efficiency compared with other traditional methods. However, current AIM methods omit discussions on the practical challenge of CAV heterogeneity. As CAVs exhibit different dynamics features and perception capabilities, it is inappropriate to adapt identical control schemes. Besides, existing MARL methods that lack heterogeneity adaptability may experience a performance decline. In response, this paper exploits MARL to model the decision-making process among CAVs and proposes a novel heterogeneous-agent attention gated trust region policy optimization (HAG-TRPO) method. The proposed method can accomplish more effective and efficient AIM with CAV discrepancies by applying a sequential update schema that boosts the algorithm adaptability for MARL tasks with agent-level heterogeneity. In addition, the proposed method utilizes the attention mechanism to intensify vehicular cognition on disordered ambience messages, as well as a gated recurrent unit for temporal comprehension on global status. Numerical experiments verify that our method results in CAVs passing at the intersection with fewer collisions and faster traffic flow, showing the superiority of our method over existing benchmarks in terms of both traffic safety and efficiency.

I. Introduction

As an essential scenario in intelligent transportation systems, autonomous intersection management (AIM) enables efficient and automatic scheduling for connected and autonomous vehicles (CAVs) without rigid signal controls. However, it remains several difficulties for resolving vehicular decision-making in versatile intersection ambiences.