MIMIC-Pose: Implicit Membership Discrimination of Body Joints for Human Pose Estimation

The objective of human pose estimation is to accurately identify the positions of individual body joints for each person seen in an image for human kinematics modelling and analysis. A majority of current deep-learning approaches focus on utilizing feature learning to regress the coordinates of each individual joint. This can be thought of as a constrained point distribution optimization problem in the image plane. However, in the high-dimensional feature space, the feature distribution of joints as a significant constraint and discriminative condition, has not received enough attention yet. Here we propose a novel approach (MIMIC-Pose) that applies an implicit pair-wise keypoint membership constraint using those features to individual joint regression to ensure that the joints of the same person have higher mutual feature similarities compared with those of different people or the image backgrounds. We propose a novel link prediction integrated learning architecture with high-throughput tokens to learn such similarity feature embeddings to improve the prediction accuracy of individual joints in a single forward inference. Experimental results show that our approach can achieve competitive performance with much fewer model parameters and lower computational cost compared to state-of-the-art methods.

I. Introduction

Human pose estimation is a useful computer vision technique in empowering a wide range of real-world intelligent applications, such as smart sports [8], healthcare [25], and human behavior cognition [24]. Typical current human pose estimation techniques often solve this problem by transforming the discrete 2D coordinates of human keypoints into heatmaps and training a deep neural network to predict the heatmaps [4], [38], [7], [33], [16], [17]. This process can be seen as a constrained point distribution optimization problem in the image plane [27]. However, this paradigm solely focuses on the prediction of each individual body joint from an input image and fails to consider the inference of body parts of the same person as a cohesive unit, which should be accurately discriminated from those of other people in the image. This can lead to inaccurate assignment of body joints to individuals in challenging scenarios.