Design of Intelligent Connected Cruise Control With Vehicle-to-Vehicle Communication Delays

Connected cruise control (CCC) refers to a type of advanced driver assistance system combined with wireless vehicle-to-vehicle (V2V) communication technology to improve control stability and driving safety. However, it is urgent to investigate intelligent control algorithms to improve the adaptability of CCC in complex traffic environment. In addition, the effect of communication delays attributed to shared wireless communications on the performance degradation of the intelligent CCC design cannot be ignored. In this study, the design of deep reinforcement learning (DRL) controller for CCC system in high-dynamic traffic scenarios is investigated, which considers both time-varying leading velocity and communication delays. To be more specific, an intelligent CCC algorithm based on deep deterministic policy gradient (DDPG) is developed. According to the training samples obtained from interacting with the traffic environment, the actor network and the critic network are trained to maximize the quadratic reward function determined by state errors and control inputs for generating intelligent control strategies. In particular, the effect of previous control strategies is considered in vehicle dynamics analysis, optimization problem formulation and Markov decision process (MDP) definition to compensate the performance degradation attributed to communication delays. Lastly, the effectiveness and convergence of the proposed DRL-based CCC controller are verified through numerical simulations with various conditions. The superior performance of the proposed algorithm is shown by comparing with existing traditional algorithms and state-of-the-art DRL algorithms.

I. Introduction

Traffic congestion and driving safety have become significant issues hindering the rapid urban development over the past few years [1]. The defects of human-driven vehicles (e.g., limited perception range and long reaction time) have not yet been solved though traffic demand management, dynamic traffic control and other traffic automatic solutions have been developed. Accordingly, advanced driver assistance systems (ADAS), as an effective way to reduce traffic congestion and improve driving safety, have been extensively concerned [2]. Cruise control system (CCS) is a typical application of ADAS, which can help the vehicle travel autonomously at the desired velocity to alleviate the driver’s burden [3].