Multi-Agent Spatial-Temporal Transformer for Traffic Signal Control

Multi-agent Deep Reinforcement Learning (MDRL) methods have demonstrated considerable promise in the optimization of traffic signal control (TSC) in road networks. Nevertheless, these methods are not without their limitations, including an increased level of complexity. In order to mitigate traffic congestion more effectively, multi-agent spatial-temporal transformer (ST-MAT) TSC model based on MDRL was proposed in this paper which employs a sequence model. The inclusion of the spatial-temporal attention mechanism in this model allows for the deduction of the spatial-temporal correlation between agents in the environment. This mechanism aids agents in dynamically perceiving the traffic environment and determining the importance of crossings. This study employs experimental trials conducted on the 4×4 synthetic road network and the real road networks of Ingolstadt. The results indicate that the ST-MAT model, when implemented in both simulation environments, successfully mitigates traffic congestion by reducing the average waiting time of vehicles to 14s and 40s, respectively. The utilization of the spatial-temporal attention mechanism enables agents to rapidly adjust to various traffic environments. The model shows commendable applicability in TSC tasks.