Platoon Control of Connected Vehicles via Safe Reinforcement Learning Based on Lyapunov Based Soft Actor Critic Algorithm

This paper addresses the platoon control problem of connected vehicular systems subject to safety constraints. A constrained Markov Decision Process model is established. A novel safe-reinforcement-learning control method based on the Lyapunov-based Soft Actor-Critic (LSAC) algorithm is developed, where the LSAC algorithm is designed to maximize the reward while minimizing the safety cost, ensuring minimal risk of collisions. Comparisons between the safe-reinforcement-learning control via LSAC algorithm and Soft Actor-Critic (SAC) algorithm are conducted to demonstrate the effectiveness of the proposed algorithm.

I. Introduction

Significant progress has been achieved in cooperative platooning of connected autonomous vehicular systems [1]. Benefiting from various onboard sensors and vehicle-to-vehicle (V2V) communication technique, vehicles can exchange information such as the speed, acceleration and position between each other. Cooperative platooning of autonomous vehicular systems aims to design a distributed control law such that vehicles can travel at the same speed while maintaining a desired inter-vehicle distance. Therefore, the vehicular platooning technique can effectively reduce fuel consumption while simultaneously increasing road capacity. However, in practice, vehicles will be subject to various constraints to keep safety, such as the safe inter-vehicle distances, input saturation, and so on. It is still a challenge to cooperatively learn distributed control strategy for a connected autonomous vehicular system subject to multiple constraints.