Reinforcement Learning-Based Trajectory Planning and Tracking Control for Connected Vehicles Subject to Communication Delay

To address the vehicle trajectory planning and tracking control problem in vehicular platoon control, we propose a two-layered hierarchical control framework based on deep deterministic policy gradient (DDPG) combined with incremental PID with switching function, considering the communication abnormal (packet-dropout, time delay, interruption) during the following process. Firstly, the vehicle dynamics model is constructed based on vehicle theory, secondly, the state space, action space, and multi-objective reward function are designed based on reinforcement learning theory for the decision-making of the RL-Agent, and finally, the effectiveness of the designed controller was validated through experimental simulations. The results show that: the controller designed in this paper, not only can realize the vehicle speed error, and vehicle space error quickly converge to zero, but also the communication time delay, the following vehicle can still smoothly follow the preceding vehicle, acceleration, and speed.