DiffusionRegPose: Enhancing Multi-Person Pose Estimation Using a Diffusion-Based End-to-End Regression Approach

This paper presents the DiffusionRegPose, a novel approach to multi-person pose estimation that converts a one-stage, end-to-end keypoint regression model into a diffusion-based sampling process. Existing one-stage deterministic re-gression methods, though efficient, are often prone to missed or false detections in crowded or occluded scenes, due to their inability to reason pose ambiguity. To address these challenges, we handle ambiguous poses in a generative fashion, i.e., sampling from the image-conditioned pose distributions characterized by a diffusion probabilistic model. Specifically, with initial pose tokens extracted from the image, noisy pose candidates are progressively refined by inter-acting with the initial tokens via attention layers. Extensive evaluations on the COCO and CrowdPose datasets show that DiffusionRegPose clearly improves the pose accuracy in crowded scenarios, as evidenced by a notable 4. 0 AP in-crease in the APH metric on the CrowdPose dataset. This demonstrates the model's potential for robust and precise human pose estimation in real-world applications. Code will be available at https://github.com/cici203IDiffusionRegPose.

1. Introduction

Multi-person pose estimation is a well-explored area in computer vision, which involves locating the keypoints that cor-respond to body parts of each person within an image. It has been adopted in various applications, including human action recognition [39], [40], human body reconstruction [35], [47], and human image generation [15], [21]. The multi-person pose estimation can be broadly classified into three categories: top-down [26], [34], [36], [42], [44], bottom-up [5], [8], [22], [38], and one-stage methods [20], [29], [37], [41]. The top-down method typically relies on an off-the-shelf object detector, which is firstly applied to identify persons within an image and then followed by the single-person pose estimation. In contrast’ the bottom-up approach initially detects keypoints in an instance-agnostic manner and subsequently groups them to form individual human instances. Compared to those two-stage approaches above, the one-stage method is capable of directly outputting a sequence of potential human poses, yielding improved computing efficiency and thus drawing increased research attention. Figure 1.

Comparison of one-stage end-to-end human pose estimation frameworks. The naive method is represented in (a), while our proposed diffusion-based approach is illustrated in (b).