Naive Data Augmentation Might Be Toxic: Data-Prior Guided Self-Supervised Representation Learning for Micro-Gesture Recognition

Body gestures play an important role in nonverbal communication because they transmit emotional information. Recently, a specific group of gestures, so-called Micro-gestures (MGs), has drawn increasing research interests in the community, as they can be useful cues to interpret human inner feelings. In this study, we focused on recognizing MG via self-supervised learning from skeleton sequences with several contributions. Initially, we observed that existing data augmentation methods for skeleton data always fail in MG representation learning. Our investigation shows that the failure is caused by the inherent properties of real-world datasets, such as imbalanced/long-tail data distribution, intra-class ambiguity, and inter-class heterogeneity. Thus, we propose a novel prior-guided augmentation strategy that can preserve the original data distribution while maximizing the agreement between samples in self-supervised learning. Furthermore, we proposed a three-stream architecture of self-supervised presentation learning for micro-gestures via spatial/temporal masking to jointly enhance the learning of invariant features. Lastly, the experimental results show that our proposed method has achieved state-of-the-art performances on two public MG datasets.