Learned Dual-View Reflection Removal

Traditional reflection removal algorithms either use a single image as input, which suffers from intrinsic ambiguities, or use multiple images from a moving camera, which is inconvenient for users. We instead propose a learning-based dereflection algorithm that uses stereo images as input. This is an effective trade-off between the two extremes: the parallax between two views provides cues to remove reflections, and two views are easy to capture due to the adoption of stereo cameras in smartphones. Our model consists of a learning-based reflection-invariant flow model for dual-view registration, and a learned synthesis model for combining aligned image pairs. Because no dataset for dual-view reflection removal exists, we render a synthetic dataset of dual-views with and without reflections for use in training. Our evaluation on an additional real-world dataset of stereo pairs shows that our algorithm outperforms existing single-image and multi-image dereflection approaches.

1. Introduction

Of the billions of pictures taken every year, a significant portion are taken through a reflective surface such as a glass window of a car or a glass case in a museum. This presents a problem for the photographer, as glass reflects some of the incident light from the same side as the photographer back towards the camera, corrupting the captured images with reflected image content. Formally, the captured image I is the sum of the image being transmitted through the glass T and the image of the light being reflected by the glass R:

$$\begin{equation*}I[x,y,c] = T[x,y,c] + R[x,y,c]\tag{1}\end{equation*}$$