CDRP3: Cascade Deep Reinforcement Learning for Urban Driving Safety With Joint Perception, Prediction, and Planning

Safe urban driving is challenging due to the high density of traffic flow and various potential hazards, such as the sudden appearance of unknown objects. Traditional rule-based approaches and imitation learning methods struggle to address the diverse driving scenarios encountered in urban environments. Reinforcement learning (RL), which adapts to a wide range of driving scenarios through continuous interaction with the environment, has demonstrated success in autonomous driving. Making safety decisions when driving in urban environments necessitates a comprehensive perception of the current scene and the ability to predict the evolution of the dynamic scene. In this paper, we present a novel cascade deep reinforcement learning framework, CDRP3, designed to enhance the safety decision-making capabilities of self-driving vehicles in complex scenarios and emergencies. We leverage a multi-modal spatio-temporal perception (MmSTP) module to fuse multi-modal sensor data and introduce temporal perception to capture spatio-temporal information about dynamic driving environments, and a future state prediction (FSP) module to model complex interactions between different traffic participants and explicitly predict their future states. Subsequently, in the PPO-based planning module, we use the comprehensive environmental information obtained from perception and prediction to decode an optimized driving strategy using a lateral and longitudinal separated multi-branch network structure guided by a customized reward function. This approach enables knowledge transfer from the perception and prediction components to planning, and planning-oriented enhancement of safety decision-making capabilities to improve driving safety. Our experiments demonstrate that CDRP3 outperforms state-of-the-art methods, providing superior driving safety in complex urban environments.

I. Introduction

In recent years, significant strides have been made in autonomous driving research, yet the realization of autonomous driving in urban environments remains one of the most formidable challenges [1]. Urban traffic is inherently complex and uncertain, characterized by a multitude of traffic participants exhibiting diverse behaviors, such as sudden stops by vehicles, unexpected pedestrian crossings, and other emergencies [2]. In such settings, traditional rule-based approaches [3], [4] often fail to address all possible driving situations, potentially resulting in unsafe decisions that can lead to traffic accidents.