Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review

Behaviour prediction function of an autonomous vehicle predicts the future states of the nearby vehicles based on the current and past observations of the surrounding environment. This helps enhance their awareness of the imminent hazards. However, conventional behavior prediction solutions are applicable in simple driving scenarios that require short prediction horizons. Most recently, deep learning-based approaches have become popular due to their promising performance in more complex environments compared to the conventional approaches. Motivated by this increased popularity, we provide a comprehensive review of the state-of-the-art of deep learning-based approaches for vehicle behavior prediction in this article. We firstly give an overview of the generic problem of vehicle behavior prediction and discuss its challenges, followed by classification and review of the most recent deep learning-based solutions based on three criteria: input representation, output type, and prediction method. The article also discusses the performance of several well-known solutions, identifies the research gaps in the literature and outlines potential new research directions.

I. Introduction

Adoption of autonomous vehicles in the near future is expected to reduce the number of road accidents and improve road safety [1]. However, for safe and efficient operation on roads, an autonomous vehicle should not only understand the current state of the nearby road-users, but also proactively anticipate their future behaviour. One part of this general problem is to predict the behaviour of pedestrians (or generally speaking, the vulnerable road-users), which is well-studied in computer vision literature [2]–[5]. There are also several review articles on pedestrian behaviour prediction such as [6]–[8]. Another equally important part of the problem is prediction of the intended behaviour of other vehicles on the road. In contrast to pedestrians, vehicles’ behaviour is constrained by their higher inertia, driving rules and road geometry, which could help reduce the complexity of the problem, compared to aforementioned problem. Nonetheless, new challenges arise from interdependency among vehicles behaviour, influence of traffic rules and driving environment, and multimodality of vehicles behaviour. Practical limitations in observing the surrounding environment and the required computational resources to execute prediction algorithms also add to the difficulty of the problem, as explained in the later sections of this article.