Radial Distortion in Face Images: Detection and Impact

Acquiring face images of sufficiently high quality is important for online ID and travel document issuance applications using face recognition systems (FRS). Low-quality, manipulated (intentionally or unintentionally), or distorted images degrade the FRS performance and facilitate documents’ misuse. Securing quality for enrolment images, especially in the unsupervised self-enrolment scenario via a smartphone, becomes important to assure FRS performance. In this work, we focus on the less studied area of radial distortion (a.k.a., the fish-eye effect) in face images and its impact on FRS performance. We introduce an effective radial distortion detection model that can detect and flag radial distortion in the enrolment scenario. We formalize the detection model as a face image quality assessment (FIQA) algorithm and provide a careful inspection of the effect of radial distortion on FRS performance. Evaluation results show excellent detection results for the proposed models, and the study on the impact on FRS uncovers valuable insights into how to best use these models in operational systems.

1. Introduction

Face recognition systems (FRS) are being deployed at a number of airports, border crossings, and security gates. They have become a cornerstone of our security architecture to verify identity. Online ID and travel document issuance applications that are mobile/smartphone-compatible are being rolled out in many countries . These applications allow individuals to self-enroll by uploading an existing face photo or taking a selfie, which will then be used on the issued document as a reference image to verify the identity of the individual in a later encounter. The reliability of these systems and the reference data stored therein is therefore of great importance. FRS needs to not only perform well but also be quality-aware and immune to various kinds of attacks. Many studies have shown that low-quality images have a negative impact on FRS performance [23], [27], [3], and further, they have a negative impact on detecting presentation attacks (also morphing) [2], [13]. Robust methods that can examine the quality of the incoming images are therefore needed to flag any non-complaint or suspected image, making sure the obtained images are as representative as possible of the individual to whom they belong. According to the ISO/IEC 29794-5 standard [23], the quality assessment of a face image belongs to capture-related and subject-related quality components. Capture-related quality components such as background uniformity, illumination uniformity, under/over exposure, and natural color, and subject-related quality components such as eyes open, mouth closed, head size, and pose have dedicated metrics to measure them. Radial distortion is yet another capture-related quality component emerging due to the optical properties of the camera. Commonly known as the fish-eye effect, it distorts the captured image, creating a hemispherical (panoramic) effect. The disturbance effect on the image and the perception of the represented face are shown in the example images in Figure 1. Such an image distortion is an effect caused by user interaction that is not compliant with the capture process policy (for example, the smartphone is too close to the subject) and is not necessarily an attack. The standard ISO/IEC 19794-5 [20] mandates the "fish eye" effect associated with wide-angle lenses to be absent. Further, the standard ISO/IEC 39794-5 [21] requires the maximum magnification distortion rate to be equal to or below 7%. However, to the best of our knowledge, an algorithm to detect or estimate radial distortion in a face image is not yet contained in the draft quality testing version of the implementation-focused standard ISO/IEC 29794-5 [23].