Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections

In this work, we present a reward-driven automated curriculum reinforcement learning approach for interaction-aware self-driving at unsignalized intersections, taking into account the uncertainties associated with surrounding vehicles (SVs). These uncertainties encompass the uncertainty of SVs’ driving intention and also the quantity of SVs. To deal with this problem, the curriculum set is specifically designed to accommodate a progressively increasing number of SVs. By implementing an automated curriculum selection mechanism, the importance weights are rationally allocated across various curricula, thereby facilitating improved sample efficiency and training outcomes. Furthermore, the reward function is meticulously designed to guide the agent towards effective policy exploration. Thus the proposed framework could proactively address the above uncertainties at unsignalized intersections by employing the automated curriculum learning technique that progressively increases task difficulty, and this ensures safe self-driving through effective interaction with SVs. Comparative experiments are conducted in Highway_Env, and the results indicate that our approach achieves the highest task success rate, attains strong robustness to initialization parameters of the curriculum selection module, and exhibits superior adaptability to diverse situational configurations at unsignalized intersections. Furthermore, the effectiveness of the proposed method is validated using the high-fidelity CARLA simulator.

I. Introduction

In recent years, autonomous driving technologies have achieved notable advancements and remarkable strides in both academic and industrial domains [1] - [3]. Nevertheless, self-driving tasks in dense and interaction-heavy scenarios remain an open challenge, primarily stemming from the lack of precise driving intention of surrounding vehicles (SVs) [4], [5]. Apparently, inaccurate assessments regarding the driving intention of SVs could significantly hinder safe motion planning and potentially lead to traffic accidents [6]. The situation becomes even more severe when it comes to unsignalized intersections, where the ego vehicle (EV) needs to coordinate simultaneously with multiple vehicles approaching from various directions [7]. In general, these intersections exhibit unpredictable traffic patterns, and uncertainties in the driving behavior of SVs pose serious safety concerns during the interaction process, making it challenging for autonomous vehicles to anticipate and respond appropriately to potential hazards. In this sense, unsignalized intersections typically require complex decision-making and situational awareness.