Multi-Agent Long-Term 3D Human Pose Forecasting via Interaction-Aware Trajectory Conditioning

Human pose forecasting garners attention for its diverse applications. However, challenges in modeling the multi-modal nature of human motion and intricate interactions among agents persist, particularly with longer timescales and more agents. In this paper, we propose an interaction-aware trajectory-conditioned long-term multiagent human pose forecasting model, utilizing a coarse-to-fine prediction approach: multi-modal global trajectories are initially forecasted, followed by respective local pose forecasts conditioned on each mode. In doing so, our Trajectory2Pose model introduces a graph-based agent-wise interaction module for a reciprocal forecast of local motion-conditioned global trajectory and trajectory-conditioned local pose. Our model effectively handles the multi-modality of human motion and the complexity of long-term multiagent interactions, improving performance in complex environments. Furthermore, we address the lack of long-term (6s+) multiagent (5 +) datasets by constructing a new dataset from real-world images and 2D annotations, enabling a comprehensive evaluation of our proposed model. State-of-the-art prediction performance on both complex and simpler datasets confirms the generalized effectiveness of our method. The code is available at https://github.com/Jaewoo97/T2P.

1. Introduction

Human pose forecasting aims to predict future human motion based on observed past motion [20], [31], [32], [35], [37], [53], [86]. Humans instinctively perform such tasks, allowing them to naturally navigate in crowded areas or identify and circumvent potential dangers. For this reason, human pose forecasting plays an important role in various computer vision tasks [21], [23], [27], [54], [85], [91]. Indeed, recent years have seen a proliferation of work on multi agent motion forecasting which aim towards modeling complex multiagent inter-Although various methods have been proposed, they share two major limitations. The first is a limitation on long-term predictions, as previous studies predicted up to 3 seconds at most [4], [47], [74], [75]. However, a sufficiently long forecast horizon is essential to fully leverage human pose forecasting for diverse downstream tasks in the scope of identifying potential danger or understanding human be-havior. The second is that multi-person interactions are not proficiently learned. Existing methods consider the joints of multiple people all at once as objects of interaction [47], [65], [74], resulting in an excessive complexity with respect to the number of joints. Due to such inefficient modeling, these approaches are found to be incompetent in long-term (3s+) multiagent (6+) settings, limiting their practicality on complex real-world environments. action [20], [47], [53], [71], [74]. Figure 1.

Human motion is goal-directed and influenced by other entities. Therefore, global intention contains hints for local intention, allowing us to infer local pose from global trajectories. Our method first forecasts global trajectories, upon which local poses are conditioned for subsequent forecasts. Pose and trajectory-wise interagent interactions are considered for both predictions.