Learning a Safe Driving Policy for Urban Area from Real-World Data

Driving safely on the urban roads is a major impediment in achieving level 5 autonomy. To achieve this, two main streams of approaches have been proposed: module-based and end-to-end. Module based solutions try to solve the problem by dividing the whole task of driving into separate modules and solving each one at a time. On the other hand, end-to-end approaches try to provide the control command directly from the sensor data input, like what a human driver does. Deep reinforcement learning (DRL) is one of the algorithm families that has received much attention recently to achieve end-to-end solutions. As this approach is challenging, almost all the related works use simulator generated data for training a policy network. However, synthetic data does not capture the complexity, variability, realism, and diversity of the real-world environment. A reinforcement learning (RL) policy trained on synthetic dataset necessarily makes it unreliable in real-world deployment. In this study, we propose an actor-critic DRL model to learn a driving policy from a real-world urban driving dataset. The policy enables the RL agent to keep safe distance from the leading vehicle, follow traffic light, and prevents the agent from going off-road. To optimize the policy we use proximal policy optimization (PPO), a state-of-the-art reinforcement learning algorithm. Simulation results show that the agent learns some of the basic safe driving requirements effectively.