Accidental Light Probes

Recovering lighting in a scene from a single image is a fundamental problem in computer vision. While a mirror ball light probe can capture omnidirectional lighting, light probes are generally unavailable in everyday images. In this work, we study recovering lighting from accidental light probes (ALPs)-common, shiny objects like Coke cans, which often accidentally appear in daily scenes. We propose a physically-based approach to model ALPs and estimate lighting from their appearances in single images. The main idea is to model the appearance of ALPs by photogram-metrically principled shading and to invert this process via differentiable rendering to recover incidental illumination. We demonstrate that we can put an ALP into a scene to allow high-fidelity lighting estimation. Our model can also recover lighting for existing images that happen to contain an ALP**Project website: https://kovenyu.com/ALP. I'd rather be Shiny. - Tamatoa from Moana, 2016

1. Introduction

Traditionally, scene lighting has been captured through the use of light probes, typically a chromium mirror ball; their shape (perfect sphere) and material (perfect mirror) allow for a perfect measurement of all light that intersects the probe. Unfortunately, perfect light probes rarely appear in everyday photos, and it is unusual for people to carry them around to place in scenes. Fortunately, many everyday objects share the desired properties of light probes: Coke cans, rings, and thermos bottles are shiny (high reflectance) and curved (have a variety of surface normals). These objects can reveal a significant amount of information about the scene lighting, and can be seen as imperfect “accidental” light probes (e.g., the Diet Pepsi in Figure 1). Unlike perfect light probes, they can easily be found in casual photos or acquired and placed in a scene. In this paper, we explore using such everyday, shiny, curved objects as Accidental Light Probes (ALPs) to estimate lighting from a single image. Figure 1.

(Left) From an image that has an accidental light probe (a Diet Pepsi can), we insert a virtual object (a Diet Coke can) with estimated lighting using the accidental light probe (Middle), and using estimated lighting from a recent state-of-the-art lighting estimation method [49] (Right). Note how our method better re-lights the inserted can to produce an appearance consistent with the environment (e.g., the highlight reflection and overall intensity).