Towards High-Quality Specular Highlight Removal by Leveraging Large-Scale Synthetic Data

This paper aims to remove specular highlights from a single object-level image. Although previous methods have made some progresses, their performance remains somewhat limited, particularly for real images with complex specular highlights. To this end, we propose a three-stage network to address them. Specifically, given an input image, we first decompose it into the albedo, shading, and specular residue components to estimate a coarse specular-free image. Then, we further refine the coarse result to alleviate its visual artifacts such as color distortion. Finally, we adjust the tone of the refined result to match the tone of the input as closely as possible. In addition, to facilitate network training and quantitative evaluation, we present a large-scale synthetic dataset of object-level images, covering diverse objects and illumination conditions. Extensive experiments illustrate that our network is able to generalize well to unseen real object-level images, and even produce good results for scene-level images with multiple background objects and complex lighting.

1. Introduction

Specular highlights are very common in the real world, but they are usually undesirable in photographs, since they can degrade the image quality. In daily life, users often want to achieve the specular-free image from an image. For example, specular highlights in facial or document images sweep away skin details or meaningful texture patterns which are very important to users. Removing specular highlights from a single image enables recovering visual content with better perceptibility. Moreover, it has many related applications such as recoloring [1], light source estimation [11], recognition of specular objects [18], and intrinsic image decomposition [31]. Thus, specular highlight removal is a long-standing and challenging problem in computer vision and computer graphics.