Blind Face Restoration for Under-Display Camera via Dictionary Guided Transformer

By hiding the front-facing camera below the display panel, Under-Display Camera (UDC) provides users with a full-screen experience. However, due to the characteristics of the display, images taken by UDC suffer from significant quality degradation. Methods have been proposed to tackle UDC image restoration and advances have been achieved. There are still no specialized methods and datasets for restoring UDC face images, which may be the most common problem in the UDC scene. To this end, considering color filtering, brightness attenuation, and diffraction in the imaging process of UDC, we propose a two-stage network UDC Degradation Model Network named UDC-DMNet to synthesize UDC images by modeling the processes of UDC imaging. Then we use UDC-DMNet and high-quality face images from FFHQ and CelebA-Test to create UDC face training datasets FFHQ-P/T and testing datasets CelebA-Test-P/T for UDC face restoration. We propose a novel dictionary-guided transformer network named DGFormer. Introducing the facial component dictionary and the characteristics of the UDC image in the restoration makes DGFormer capable of addressing blind face restoration in UDC scenarios. Experiments show that our DGFormer and UDC-DMNet achieve state-of-the-art performance.

I. Introduction

In recent years, the technique of Under-Display Camera (UDC) has been widely used in smart devices, which can provide a better experience for users. For example, the smartphone using UDC can achieve true full screen without punch holes or notches. Laptops or tablets that place cameras under the center of the displays can enable a more natural gaze to focus when people use the camera. However, placing the camera below the display (e.g., OLED screen) can seriously degrade the quality of the images, e.g., blur [1], noise [2], low light [3], etc. Enhancing imaging quality can be achieved through the redesign of the display panel’s physical structure, including optimizations in the spatial arrangement of the opening. However, the development costs associated with the refining of OLED display panels can be prohibitively high. An alternative and cost-effective approach involves harnessing deep learning techniques for the restoration of images captured via Under-Display Cameras (UDC). Consequently, image restoration beneath UDC has emerged as a prominent focal point within the realm of computer vision.