Network Clustering-Based Multi-Agent Reinforcement Learning for Large-Scale Traffic Signal Control

Reinforcement learning (RL) has proven successful in the field of traffic signal control (TSC) within urban traffic networks. Moreover, multi-agent RL (MARL) achieves large-scale TSC by distributing the global control to each local RL agent. However, as the scale of joint TSC continues to expand, MARL faces greater convergence challenges. In this paper, a method combining network clustering and MARL is proposed to address this issue. We apply the Normalized Cut (Ncut) algorithm to effectively partition a large-scale traffic network, and then adopt MADQN to complete TSC tasks for each subnetwork. Meanwhile, a novel reward function based on impedance has been adopted. The results demonstrate that our method has high control performance and could effectively alleviate congestion in large-scale network under peak-hour traffic conditions.