Pedestrian Trajectory Prediction Using RNN Encoder-Decoder with Spatio-Temporal Attentions

Pedestrian motion are inherently multi-modal in nature influenced by presence of other human and physical objects in the environment. Traiectory prediction models need to address both human-human and human-space interaction issues. In this work, we leverage both pedestrians information and scene information of the navigation environment for jointly predicting trajectories of the pedestrian. We introduce a new Recurrent Neural Network based sequence model with attention mechanisms that address both human-human and human-space interaction challenges. The encoder encodes all the pedestrian trajectories and create a social context. The scene information of navigation environment is extracted using CNN and serves as a physical context for the model. Our approach utilizes physical and social attention mechanism to find semantic alignments between encoder and decoder. The social attention mechanism allow the model to look into similar step of pedestrian trajectory. The physical attention mechanism tells the model where and what to focus on the scene. Experiment on several datasets shows that the proposed approach which combine social and physical attention performs better than when this information is utilized independently.