A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation

3D human pose data collected in controlled laboratory settings present challenges for pose estimators that generalize across diverse scenarios. To address this, domain generalization is employed. Current methodologies in do-main generalization for 3D human pose estimation typically utilize adversarial training to generate synthetic poses for training. Nonetheless, these approaches exhibit several limitations. First, the lack of prior information about the target domain complicates the application of suitable augmentation through a single pose augmentor, affecting generalization on target domains. Moreover, adversarial training's discriminator tends to enforce similarity between source and synthesized poses, impeding the exploration of out-of-source distributions. Furthermore, the pose estimator's op-timization is not exposed to domain shifts, limiting its over-all generalization ability. To address these limitations, we propose a novel frame-work featuring two pose augmentors: the weak and the strong augmentors. Our framework employs differential strategies for generation and discrimination processes, facilitating the preservation of knowledge related to source poses and the exploration of out-of-source distributions without prior information about target poses. Besides, we leverage meta-optimization to simulate domain shifts in the optimization process of the pose estimator, thereby improving its generalization ability. Our proposed approach significantly outperforms existing methods, as demonstrated through comprehensive experiments on various benchmark datasets. Our code will be released at https://github.com/davidpengucf/DAF-DG.

1. Introduction

3D human pose estimation (HPE) is the process of pre-dicting the 3D coordinates of human joints from images or videos. It serves as the foundation for various applications including person re-identification [29], action recognition [10], [20], [34], human mesh recovery [44], [45], virtual reality [9], [35]. However, the annotated 3D data are often collected in controlled laboratory environments for convenience, featuring indoor settings and limited actions performed by few individuals. As a result, pose estimators trained on these labeled datasets face challenges in generalizing to varied in-the-wild scenarios. Hence, the notion of domain generalization (DG) is pivotal in incorporating knowledge from labeled (source) data into a pose estimator that could generalize well on unseen (target) data. Unlike domain adaptation (DA) which involves the training with target data, DG relies solely on the source data as a reference, without any prior information about the target data. Figure 1.

Comparisons between existing single-augmentor frame-works and our proposed dual-augmentor framework on a toy example. Current single-augmentor methods excel at simulating Target Domain 2 but exhibit limitations in simulating Target Domain 1, closely resembling the source, and Target Domain 3, deviating significantly from the source. In our framework, the weak aug-mentor excels in simulating Target Domain 1, while the strong augmentor effectively imitates both Target Domain 2 and 3.