Lens-to-Lens Bokeh Effect Transformation. NTIRE 2023 Challenge Report

We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge. Recent advancements of mobile photography aim to reach the visual quality of full-frame cameras. Now, a goal in computational photography is to optimize the Bokeh effect itself, which is the aesthetic quality of the blur in out-of-focus areas of an image. Photographers create this aesthetic effect by benefiting from the lens optical properties.The aim of this work is to design a neural network capable of converting the the Bokeh effect of one lens to the effect of another lens without harming the sharp foreground regions in the image. For a given input image, knowing the target lens type, we render or transform the Bokeh effect accordingly to the lens properties. We build the BETD using two full-frame Sony cameras, and diverse lens setups.To the best of our knowledge, we are the first attempt to solve this novel task, and we provide the first BETD dataset and benchmark for it. The challenge had 99 registered participants. The submitted methods gauge the state-of-the-art in Bokeh effect rendering and transformation.

1. Introduction

Computational photography research and recent advancements of mobile cameras aim to reach the visual quality of professional full-frame DSLR cameras [14],[23]. One of the most popular effects in photography is Bokeh, which is the aesthetic quality of the blur in out-of-focus areas of an image. This is shown in Fig. 1. In professional full-frame photography, this effect is controlled by the optical design of a lens, its aperture setting, the distance to the subject, and the focal length of the lens.