Multi-Person Pose Estimation With Accurate Heatmap Regression and Greedy Association

Multi-person pose estimation aims at localizing the 2D keypoints (or body joints) for all the people in the image. There are mainly two paradigms to perform this task: top-down and bottom-up. In this paper, we present an advanced bottom-up approach based on accurate keypoint heatmap regression and greedy keypoint association. Firstly, we develop an encoding-decoding method with Gaussian heatmaps and guiding offset fields to represent multi-person pose information, encompassing keypoint positions and adjacent keypoint associations of all individuals in the scene. In particular, we analyze the deficiency of the Gaussian heatmap representation as regards keypoint localization precision if conventional element-wise $L_{2}$ -type loss is employed merely for heatmap supervision. Therefore, we introduce a peak regularization loss to jointly supervise the heatmap regression. In addition, we present an improved Hourglass Network with multi-scale heatmap aggregation to simultaneously infer the said encoding. Finally, we propose a novel focal $L_{2}$ loss to help the network cope with the imbalanced problem of keypoint detection in heatmaps. Our results show that the proposed approach surpasses other bottom-up approaches on COCO dataset, and even outperforms the top-down approaches on CrowdPose dataset containing more crowded scenes.

I. Introduction

Human-centered pose estimation, such as gaze estimation, hand pose estimation and human body posture estimation attract much research interests [1]–[5], as they can greatly facilitate many computer vision tasks such as human-computer interaction [6], action recognition [7] and person re-identification [8]. In this paper, we concentrate on the very challenging problem of multi-person pose estimation, which aims at obtaining every 2D human pose represented by a set of anatomical keypoints (or body joints) in an image [9], [10].