Spatial-Aware Regression for Keypoint Localization

Regression-based keypoint localization shows advan-tages of high efficiency and better robustness to quantization errors than heatmap-based methods. However, existing regression-based methods discard the spatial location prior in input image with a global pooling, leading to in-ferior accuracy and are limited to single instance localization tasks. We study the regression-based keypoint localization from a new perspective by leveraging the spatiallocation prior. Instead of regressing on the pooled feature, the proposed Spatial-Aware Regression (SAR) maintains the spatial location map and outputs spatial coordinates and confidence score for each grid, which are optimized with a unified objective. Benefited by the location prior, these spatial-aware outputs can be efficiently optimized, resulting in better localization performance. Moreover, incorporating spatial prior makes SAR more general and can be applied into various keypoint localization tasks. We test the proposed method in 4 keypoint localization tasks including single/multi-person 2D/3D pose estimation, and the whole-body pose estimation. Extensive experiments demonstrate its promising performance, e.g., consistently outperforming recent regressions-based methods^††project pagn: https://github.com/kennethwdk/SAR.

1. Introduction

Keypoint localization aims to locate target keypoints from an input image and is a fundamental task in the field of computer vision. It has a wide range of applications in human pose estimation [21], [26]–[28] and facial landmark detection [19], et al. Existing methods for keypoint localization can be summarized into two categories: heatmap-based [21], [29], [31] and regression-based [10], [23], [25], respectively. Regression-based method directly adopts neural network to learn the mapping from input RGB image to key-point coordinates. Heatmap-based method uses a probability map (also referred as heatmap) to encode the likelihood of the target location and retrieves it by selecting location with the highest probability. Figure 1.

Illustration of (Top) regression-based method, (Middle) standard heatmap-based method, and (Bottom) the proposed SAR for keypoint localization.