Portrait Shadow Removal Using Context-Aware Illumination Restoration Network

Portrait shadow removal is a challenging task due to the complex surface of the face. Although existing work in this field makes substantial progress, these methods tend to overlook information in the background areas. However, this background information not only contains some important illumination cues but also plays a pivotal role in achieving lighting harmony between the face and the background after shadow elimination. In this paper, we propose a Context-aware Illumination Restoration Network (CIRNet) for portrait shadow removal. Our CIRNet consists of three stages. First, the Coarse Shadow Removal Network (CSRNet) mitigates the illumination discrepancies between shadow and non-shadow areas. Next, the Area-aware Shadow Restoration Network (ASRNet) predicts the illumination characteristics of shadowed areas by utilizing background context and non-shadow portrait context as references. Lastly, we introduce a Global Fusion Network to adaptively merge contextual information from different areas and generate the final shadow removal result. This approach leverages the illumination information from the background region while ensuring a more consistent overall illumination in the generated images. Our approach can also be extended to high-resolution portrait shadow removal and portrait specular highlight removal. Besides, we construct the first real facial shadow dataset for portrait shadow removal, consisting of 6200 pairs of facial images. Qualitative and quantitative comparisons demonstrate the advantages of our proposed dataset as well as our method.

I. Introduction

As a part of portrait editing, portrait shadow removal aims to eliminate the complex foreign shadow of the face surface and restore the high-quality facial image. It is the most fundamental and important task in the field of computer vision [1], and is widely used in our daily lives, especially in the live streaming industry and the facial recognition system. Since portraits often inevitably suffer from shadows under complex environmental lighting conditions, this seriously degrades the aesthetic quality of the face and greatly reduces the performance of some computer vision tasks, such as face recognition and detection [2]. Obviously, effectively removing portrait shadows is really an important but challenging task.