Cooperative Reinforcement Learning-based Damping of Lane-Change-Induced Waves

In this article, we demonstrate the first successful application of using reinforcement learning (RL) to develop policies for connected, automated vehicles (CAVs) to mitigate the effects of lane changing in traffic. We discuss how lane changing is a source of wave propagation and disturbance in certain kinds of traffic and propose a RL-based solution for wave damping. While receiving information from the environment and the ego vehicle (connected, non-automated) which is performing a lane change, we train an RL agent, operating as a CAV, to mitigate the waves caused by the lane change. The CAV has an advantage in being able to plan given the information of the vehicle executing the lane change, providing the CAV with anticipatory foresight as well as practical downstream information. At evaluation, the RL-based policy achieves up to a 5.3% improvement in velocity and a 15.9% improvement in throughput. It completely mitigates the formation of waves for certain inflow rates, and facilitates significant improvements for other inflow rates.