Burst Reflection Removal using Reflection Motion Aggregation Cues

Single image reflection removal has attracted lot of interest in the recent past with data driven approaches demonstrating significant improvements. However deep learning based approaches for multi-image reflection removal remains relatively less explored. The existing multi-image methods require input images to be captured at sufficiently different view points with wide baselines. This makes it cumbersome for the user who is required to capture the scene by moving the camera in multiple directions. A more convenient way is to capture a burst of images in a short time duration without providing any specific instructions to the user. A burst of images captured on a hand-held device provide crucial cues that rely on the subtle handshakes created during the capture process to separate the reflection and the transmission layers. In this paper, we propose a multi-stage deep learning based approach for burst reflection removal. In the first stage, we perform reflection suppression on the individual images. In the second stage, a novel reflection motion aggregation (RMA) cue is extracted that emphasizes the transmission layer more than the reflection layer to aid better layer separation. In our final stage we use this RMA cue as a guide to remove reflections from the input. We provide the first real world burst images dataset along with ground truth for reflection removal that can enable future benchmarking. We evaluate both qualitatively and quantitatively to demonstrate the superiority of the proposed approach. Our method achieves ~ 2dB improvement in PSNR over single image based methods and ~ 1dB over multi-image based methods.

1. Introduction

Many modern day cameras, especially those present in smart phones have shown significant advances in terms of achieving good image quality in different lighting conditions. Low level computational imaging tasks such as image de-noising [7], high dynamic range (HDR) imaging [10] [12], have shown tremendous improvements in the recent past with the advent of data driven approaches. However, high-level computational imaging tasks such as image inpainting, removing obstructions, reflections, shadows, etc., still pose significant challenges in terms of achieving acceptable image quality. More recently, deep learning based approaches have shown tremendous amount of progress in reflection removal [30] [35] [34] [14] [22] compared to the traditional computationally expensive optimization based methods [25] [16] [17] making them a viable choice to be deployed on consumer products such as smart phones.