Stealthy Black-Box Attack With Dynamic Threshold Against MARL-Based Traffic Signal Control System

Multiagent reinforcement learning (MARL) promises outstanding performance for multiintersection traffic signal control systems (TSCS), enabling intelligent administration of cities. However, the vulnerability of MARL algorithms to adversarial attacks has raised concerns about the security of TSCS. In this article, we explore the robustness of MARL-based TSCS against adversarial attacks, propose a black-box multiobject attack strategy, and assign an attack budget to ensure stealthiness. We design a dynamic threshold-based selection of critical states to minimize the cumulative reward with a limited number of attacks. In addition, we present a lightweight agnostic dynamic threshold-based defense mechanism by enhancing the worst-case performance of the policy. We formulate it as a min-max optimization problem, i.e., minimizing the quantity of training sample alterations while maximizing the cumulative discount reward of policy against the perturbed states. Extensive experiments on simulation of urban mobility (SUMO) demonstrate that the proposed attack policy can significantly reduce the performance of TSCS.

I. Introduction

Urban traffic congestion keeps growing severely, which has hurt the global economy. For instance, in 2018, the United States suffered more than $87 billion in losses due to traffic congestion [1]. Intelligent transportation systems based on state-of-the-art technology, including automated control, smart sensing, and artificial intelligence, develop rapidly to ease the burden of urban traffic congestion [2], [3], [4]. In particular, the cooperative strategy of traffic signal control systems (TSCS) at multiple intersections is crucial in ensuring urban traffic operations' efficiency.