Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning

Based on the assumption that low-resolution (LR) and high-resolution (HR) manifolds are locally isometric, the neighbor embedding super-resolution algorithms try to preserve the geometry (reconstruction weights) of the LR space for the reconstructed HR space, but neglect the geometry of the original HR space. Due to the degradation process of the LR image (e.g., noisy, blurred, and down-sampled), the neighborhood relationship of the LR space cannot reflect the truth. To this end, this paper proposes a coarse-to-fine face super-resolution approach via a multilayer locality-constrained iterative neighbor embedding technique, which intends to represent the input LR patch while preserving the geometry of original HR space. In particular, we iteratively update the LR patch representation and the estimated HR patch, and meanwhile an intermediate dictionary learning scheme is employed to bridge the LR manifold and original HR manifold. The proposed method can faithfully capture the intrinsic image degradation shift and enhance the consistency between the reconstructed HR manifold and the original HR manifold. Experiments with application to face super-resolution on the CAS-PEAL-R1 database and real-world images demonstrate the power of the proposed algorithm.

I. Introduction

The past twenty years have witnessed a rapid progress in face recognition techniques. Due to the limitations of surveillance system, such as network bandwidth limitation, server storage, and long distance to the interest object, the query face images captured by surveillance camera are very Low-Resolution (LR). Based on the current technical level, the information revealed by LR face images is so limited that the accuracy of the resulting face recognition is very low. Thus the LR problem forms one of the most challenging issues in face recognition [1]. Recently, face super-resolution (or face hallucination) techniques have been employed to address the LR problems of imaging system. It can generate an High-Resolution (HR) face image from either a sequence of LR face images (by multi-frame reconstruction based super-resolution approaches) or a single frame LR face image (by learning-based super-resolution approaches), thus providing more facial details for the following recognition process. In this paper, we focus on the learning-based face super-resolution method for its superiority over the multi-frame approaches, especially when the magnification factor is large [2], [3].