NTIRE 2021 Depth Guided Image Relighting Challenge

Image relighting is attracting increasing interest due to its various applications. From a research perspective, im-age relighting can be exploited to conduct both image normalization for domain adaptation, and also for data augmentation. It also has multiple direct uses for photo montage and aesthetic enhancement. In this paper, we review the NTIRE 2021 depth guided image relighting challenge.We rely on the VIDIT dataset for each of our two challenge tracks, including depth information. The first track is on one-to-one relighting where the goal is to transform the illumination setup of an input image (color temperature and light source position) to the target illumination setup. In the second track, the any-to-any relighting challenge, the objective is to transform the illumination settings of the in-put image to match those of another guide image, similar to style transfer. In both tracks, participants were given depth information about the captured scenes. We had nearly 250 registered participants, leading to 18 confirmed team sub-missions in the final competition stage. The competitions, methods, and final results are presented in this paper.

1. Introduction

Due to its broad utility in research and in practice, image relighting is gaining increasingly more attention. An image relighting method would enable extended data augmentation, by providing images with various illumination settings, and similarly enables illumination domain adaptation as it can transform test images into a standard unique illumination setup. In practice, image relighting is also useful for photo montage and other aesthetic image retouching applications. The task of image relighting is however challenging. The relighting method needs to understand the geometry and illumination of the scene, be able to remove shadows, recast shadows, and occasionally inpaint totally dark areas, on top of the transformation of the illuminant.