Assessing face image quality for smartphone based face recognition system

In recent years, the popularity of smartphones has increased massively as a personal and authentication device. Face based biometrics is being used to secure the device and control access to several different services via smartphones such as payment gateways etc. Thus, to maintain the reliability and to obtain better verification performance, there is a need to adopt the standards recommended for face sample quality. In this paper, we present an evaluation of face image quality assessment using well-established ISO standards on the images collected using smartphones. In this work, we constructed a new database of 101 individuals with 22 frontal face images with different facial pose angles, illumination and at five different distances between the subject and the mobile device. We evaluate the existing quality metrics and further propose a new quality metric based on vertical edge density that can robustly estimate the pose variations and improves the quality estimation of a face image. The proposed method is evaluated for reliable estimation of the quality for smartphone face biometrics.

I. Introduction

Major finance applications on smartphones such as Apple-Pay and GooglePay have actively started to employ biometrics for verifying the customers. Similarly, face recognition is being used in different types of mobile applications such as mobile device security, mobile payment gateways, etc. In this kind of applications user's face image is captured in a relaxed or unconstrained environment. The number of factors such as ambient illumination, pose due to different ways of interacting with mobile device and distance of imaging results in varying quality face images. The quality factors of face images obtained using smartphones can be closely correlated to quality factors seen with traditional Face Recognition System (FRS). Hence they are prone to the similar problems confirmed by series of face recognition vendor tests such as an uncontrolled variation of illumination, pose, and age variations. These are three major problems which can reduce the performance of FRS drastically [1]–[2]–[3]. Further, the technical report ISO/IEC TR 29794-5 [4] defines different measures to observe the objective quality of an input image. These measures should be applied at the time of enrolment and if possible also for recognition attempts, to achieve optimal recognition performance. Most of the state-of-art commercial biometric systems in today's world are well equipped with quality assessment techniques to achieve good biometric performance. The technical report ISO/IEC TR 29794–1 [5] describes the methods for calculating the quality scores using different approaches such as “bottom-up”, “top-down” and “combined” manner. The proper understanding of the quality score calculations respecting the character of the source (i.e. the biometric characteristic) as well as the concepts of fidelity and utility can be achieved using the defined standards. The report I SO/IEC TR 29794-4 [6] generalizes the methodologies for fingerprint images. Further, the report ISO/IEC TR 29794-5[4] describes the methodologies for facial images to control the sample quality during the enrolment process for many of the commercial applications. It also gives insights about the calculations of pose and illumination symmetry of the input image. In the prior studies on face image quality, most of the work is based on image properties such as brightness, contrast, and sharpness, etc. [7]. In [8] the authors have proposed methods for illumination and pose calculations which are also adopted in ISO/IEC TR 29794-5 [4]. The quality of biometric images using different image degradations is evaluated in [9]. Further, in [10] authors have proposed a novel approach to assess the face image quality for automatic border control systems. Although there are many works on facial image quality assessment operating in conventional FRS, there are no such image quality evaluations and detailed studies carried out for face samples captured using smartphones to analyze the behavior of FRS operating on smartphone. The key contributions of this work can be outlined as: •

This paper formulates a unified framework for FRS with quality assessment, specifically for the smartphone environment to complement the increasing use of face biometrics on smartphones in large scale.

•

We first evaluate various quality metrics traditionally employed in conventional FRS [4] precisely for the smartphone based application and further propose new metric to improve the quality assessment.

•

We create a new face image database consisting of 101 subjects collected using two different smartphones to evaluate the existing metrics and newly proposed face quality metric.

Fig. 1:

Face images with different pose angles and illumination

Fig. 2:

FRS framework with quality assessment