Swarm reinforcement learning for traffic signal control based on cooperative multi-agent framework

Congestion, accidents, pollution, and many other problems resulting from urban traffic are present every day in most cities around the world. The growing number of traffic lights in intersections needs efficient control, and hence, automatic systems are essential nowadays for optimally tackling this task. Agent based technologies and reinforcements learning are largely used for modelling and controlling intelligent transportation systems, where agents represent a traffic signal controller. Each agent learns to achieve its goal through many episodes. With a complicated learning problem, it may take much computation time to acquire the optimal policy. In this paper, we use a population based methods such as particle swarm optimization to be able to find rapidly the global optimal solution for multimodal functions with wide solution space. Agents learn through not only on their respective experiences, but also by exchanging information among them, simulation results show that the swarm Q-learning surpass the simple Q-learning causing less average delay time and higher flow rate.

I. Introduction

Nowadays, most cities in the world suffer from an extreme vehicular traffic that provokes severe problems like pollution, congestion, security and many others. the construction of a new infrastructure is expensive, thus the solution is to move towards the utilization of the existing resources like intelligent transportation systems (ITS) for traffic management and control. Traffic control contains a set of technics used to improve the traffic network performance by, for example, controlling the traffic flow to minimize congestion, waiting times, fuel consumption and avoid accidents, allowing vehicles to travel more quickly through the system. Traffic control generally uses different actors, such as, controlling the traffic signals in urban areas, enforcing variable speed limits, ramp-metering in high-ways, and using driver-assistance systems (e.g., adaptive cruise control). In this paper, we particularly focus on controlling traffic signals in urban areas. The use of multi-agents systems and artificial Intelligence (AI) are becoming more and more reliable in ITS, it allows to decompose the system into multiple agents that interact with each others to achieve a desired goal, improving the ability of a traffic signal to efficiently serve vehicles at an intersection, to reduce the delay experienced by vehicular, thus increasing average network speed.