Dr.Bokeh: DiffeRentiable Occlusion-Aware Bokeh Rendering

Bokeh is widely used in photography to draw attention to the subject while effectively isolating distractions in the background. Computational methods can simulate bokeh effects without relying on a physical camera lens, but the inaccurate lens modeling in existing filtering-based meth-ods leads to artifacts that need post-processing or learning-based methods to fix. We propose Dr.Bokeh, a novel ren-dering method that addresses the issue by directly correcting the defect that violates physics in the current filtering-based bokeh rendering equation. Dr.Bokeh first preprocesses the input RGBD to obtain a layered scene representation. Dr.Bokeh then takes the layered representation and user-defined lens parameters to render photo-realistic lens blur based on the novel occlusion-aware bokeh rendering method. Experiments show that the non-learning based renderer Dr.Bokeh outperforms state-of-the-art bokeh ren-dering algorithms in terms of photo-realism. In addition, extensive quantitative and qualitative evaluations show that the more accurate lens model pushes the limit of depth-from-defocus.

1. Introduction

Bokeh is a physical effect produced by a camera lens system. It refers to the shape and quality of out-of-focus areas in an image. Bokeh brings focus to the in-focus subject and enhances the overall aesthetic quality of the image. Figure 1.

Being occlusion-aware, Dr.Bokeh renders realistic bokeh effects from the bokeh rendering process without post-processing. Compared with the scattering/gathering-based method SteReFo and learning-based method BokehMe, Dr.Bokeh renders natural partial occlusion (red parts). MPIB learns to render a partial occlusion effect but breaks on unseen data (blue parts). Dr.Bokeh is more robust than learning-based methods given the same inputs because the rendering process is physically grounded. Best viewed by zooming in.

Figure 2.

Artifacts by inaccurate lens model: color bleeding and partial occlusion are two main artifacts introduced by current inaccurate lens model. Color bleeding means the pixels in the out-of-focus scatter to in-focus regions. Partial occlusion is a semi-transparent effect on the out-of-focus boundary regions, where part of the backgrounds are visible in the background in-focus case. Best viewed by zoom-in.