Deep Reinforcement Learning With Fuzzy Feature Fusion for Cooperative Control in Traffic Light and Connected Autonomous Vehicles

A mixed traffic environment of manual driving and automatic driving will become the norm in future intelligent transportation systems. The deep reinforcement learning (DRL) method has shown significant promise in cooperative control for traffic lights and connected autonomous vehicles (CAV) in a mixed-traffic environment. However, the uncertainty and noise in integrating agents' observations can lead to inadequate exploration of environmental data by DRL algorithms. Consequently, these algorithms are prone to overfitting and becoming trapped in local optimal, which limits the performance of control strategies. To more effectively harness the gathered environmental data and thereby facilitate improved decision-making by agents, a DRL-based cooperative control method with fuzzy feature fusion (F3DRL) was proposed in this article. First, the adaptive fuzzy inference module is implemented to adaptively mitigate information uncertainty as the data from CAV is aggregated. Then, a deep information extraction module was introduced and integrated with the output of the adaptive fuzzy inference module to establish a parallel feature fusion module. The adaptive fuzzy inference module mitigates uncertainty in the extracted traffic environmental states, while the deep information extraction module facilitates the extraction of a more comprehensive environmental representation. The fusion of features derived from these two distinct modules aids DRL agents in making better action selections, which significantly enhances the effectiveness and stability of the F3DRL method. In simulations, F3DRL significantly reduced travel and delay times, fuel consumption, and CO$_{2}$ emissions, outperforming both traditional and state-of-the-art methods.

I. Introduction

Undeniably, connected autonomous vehicles (CAVs) outperform human-driven vehicles (HDVs) in many aspects, which drives the transformation of transportation systems and will have a profound impact [1]. First, CAVs may interface with nearby automobiles and roadside infrastructure to transmit real-time traffic data, including vehicle status, traffic light status, and junction geometry. Second, compared to human drivers, CAVs have quicker response times and can handle all driving-related tasks by themselves [2]. Precise trajectory control of CAVs is made feasible by these features. In the case of the fully CAV environment, CAVs even can pass through “signal-free” intersections without stopping and achieve improvement in traffic efficiency, safety, energy economics, and pollution reduction by applying platooning and coordination strategies [3], [4], [5]. However, the penetration of autonomous driving technology will be a gradual process. It is still a long way away from achieving a high level of CAV penetration or a fully CAV environment [6]. HDVs will share road resources with CAVs across long periods in the future, forming a mixed traffic environment of manual driving and automatic driving. As a result, the research on intersection control under mixed traffic with CAVs remains a hot field of traffic control theory and application around the world [7], [8], [9], [10].