Online Adaptive Temporal Memory with Certainty Estimation for Human Trajectory Prediction

Pedestrian trajectory prediction is an essential component of autonomous systems and robot navigation. Recent research has shown promising predictive performance by designing prediction networks to model a variety of motion-related features. Different from existing works, our focus is on designing a novel online adaptation framework (OAT-Mem) to exploit the temporal similarities among trajectory samples encountered during testing to improve the prediction accuracy of any such models (i.e., predictors) without knowing the details of these predictors. Our framework consists of two novel modules: an augmented temporal observation-target memory network (ATM) and a certainty-based selector (CS). Inspired by the concept of key-value memory networks [16], ATM is proposed to learn the temporal information from short-term past frames by encoding the trajectory samples of past pedestrians in form of observation-target (i.e., key-value) during testing. In addition, we propose a certainty-based selector (CS) to enhance the predictive ability of our framework under scenarios where there are large temporal dissimilarities between current pedestrians’ movements and those stored in memory. In dynamic scenes, these scenarios commonly occur due to abrupt changes in contexts, such as camera motions, scene contexts, and pedestrians’ behaviors. We extensively evaluate our framework in commonly-used datasets: JAAD [12] and PIE [19] and show that our framework significantly improves the prediction accuracy of state-of-the-art models. Finally, in-depth studies are conducted to show the importance of each proposed component.