Semi-supervised Semantics-guided Adversarial Training for Robust Trajectory Prediction

Predicting the trajectories of surrounding objects is a critical task for self-driving vehicles and many other autonomous systems. Recent works demonstrate that adversarial attacks on trajectory prediction, where small crafted perturbations are introduced to history trajectories, may significantly mislead the prediction of future trajectories and induce unsafe planning. However, few works have addressed enhancing the robustness of this important safety-critical task. In this paper, we present a novel adversarial training method for trajectory prediction. Compared with typical adversarial training on image tasks, our work is challenged by more random input with rich context and a lack of class labels. To address these challenges, we propose a method based on a semi-supervised adversarial autoencoder, which models disentangled semantic features with domain knowledge and provides additional latent labels for the adversarial training. Extensive experiments with different types of attacks demonstrate that our Semi-supervised Semantics-guided Adversarial Training (SSAT¹) method can effectively mitigate the impact of adversarial attacks by up to 73% and outperform other popular defense methods. In addition, experiments show that our method can significantly improve the system's robust generalization to unseen patterns of attacks. We believe that such semantics-guided architecture and advancement on robust generalization is an important step for developing robust prediction models and enabling safe decision making.

1. Introduction

Connected and autonomous vehicles (CAVs) have shown great promise to revolutionize the transportation systems, but also face significant concerns regarding their safety and security [15], [36], [21], [17], [50], [51]. Many of these concerns, which also apply to many other autonomous systems, arise from the increasing adoption of advanced deep neural network (DNN)-based machine learning techniques across system sensing, perception, prediction, planning, control, and general decision-making [52], [53].