Improving Car-Following Control in Mixed Traffic: A Deep Reinforcement Learning Framework with Aggregated Human-Driven Vehicles

Traffic oscillations pose safety and efficiency challenges in mixed scenarios involving connected and automated vehicles (CAVs) and human-driven vehicles (HDVs). Existing control strategies fail to handle the unpredictability of HDV behaviors, resulting in disruptive "stop-and-go" traffic patterns. This study proposes a novel algorithm that uses Deep Reinforcement Learning (DRL) integrated into a distinctive "CAV-AHDV-CAV" structure for car-following events. The consecutive HDVs are treated as an aggregated unit called Aggregated HDVs (AHDVs) to eliminate stochasticity and leverage collective traffic features as inputs, addressing the driver heterogeneity issue. Our training and testing were conducted using a dataset of 9,200 car-following events extracted from the HighD dataset. In these events, the lead vehicle serves as our CAV in front, while the following vehicle represents the AHDV. We simulated our controlled vehicle to follow the AHDV, aiming to achieve the vehicle equilibrium state with respect to both the AHDV and the CAV in front. The results demonstrate a reduction in the impact of HDVs and an enhancement of equilibrium states compared to baseline models. Specifically, we achieved a speed mean square error (MSE) of 3.151 and spacing MSE values of 50.484 (with respect to the AHDV) and 47.855 (with respect to the CAV). These findings offer robust and adaptable control strategies for efficient and safe mixed traffic dominated by CAVs.

I. Introduction

The emergence of connected and automated vehicles (CAVs) in the automotive industry has gained significant attention due to their potential to enhance traffic flow, safety, and efficiency [1], [2]. In the long term, CAVs are expected to coexist with human-driven vehicles (HDVs), creating a mixed traffic scenario [3]. However, the issue of traffic oscillations, leading to disruptive "stop-and-go" patterns, remains unresolved despite advancements in autonomous driving technology. To address this challenge, advanced control strategies such as Adaptive Cruise Control (ACC) and Cooperative Adaptive Cruise Control (CACC) have emerged [4]–[10]. These technologies offer solutions to mitigate traffic oscillations and optimize the performance of mixed traffic, enabling efficient CAV operation.