Facial Landmark Machines: A Backbone-Branches Architecture With Progressive Representation Learning

Facial landmark localization plays a critical role in face recognition and analysis. In this paper, we propose a novel cascaded backbone-branches fully convolutional neural network (BB-FCN) for rapidly and accurately localizing facial landmarks in unconstrained and cluttered settings. Our proposed BB-FCN generates facial landmark response maps directly from raw images without any preprocessing. BB-FCN follows a coarse-to-fine cascaded pipeline, which consists of a backbone network to roughly detect the locations of all facial landmarks and one branch network for each type of detected landmark to further refine its location. Furthermore, to facilitate the facial landmark localization under unconstrained settings, we propose a large-scale benchmark named SYSU16K, which contains 16 000 faces with large variations in pose, expression, illumination, and resolution. Extensive experimental evaluations demonstrate that our proposed BB-FCN can significantly outperform the state of the art under both constrained (i.e., within detected facial regions only) and unconstrained settings. We further confirm that high-quality facial landmarks localized with our proposed network can also improve the precision and recall of face detection.

I. Introduction

Facial landmark localization¹

Project website: http://www.sysu-hcp.net/facial-landmark-localization/

aims to automatically predict key point positions in facial image regions. This task is an essential component in many face-related applications, such as facial attribute analysis [1], face verification [2], [3] and face recognition [4]–[6]. Although tremendous effort has been devoted to this topic, its performance is still far from perfect, particularly on facial regions with severe occlusions or extreme head poses.