Deep Image Compositing

Image compositing is a task of combining regions from different images to compose a new image. A common use case is background replacement of portrait images. To obtain high quality composites, professionals typically manually perform multiple editing steps such as segmentation, matting and foreground color decontamination, which is very time consuming even with sophisticated photo editing tools. In this paper, we propose a new method which can automatically generate high-quality image compositing with-out any user input. Our method can be trained end-to-end to optimize exploitation of contextual and color information of both foreground and background images, where the com-positing quality is considered in the optimization. Specifically, inspired by Laplacian pyramid blending, a dense-connected multi-stream fusion network is proposed to effectively fuse the information from the foreground and back-ground images at different scales. In addition, we intro-duce a self-taught strategy to progressively train from easy to complex cases to mitigate the lack of training data. Experiments show that the proposed method can automatically generate high-quality composites and outperforms existing methods both qualitatively and quantitatively.

1. Introduction

Image compositing is one of the most popular applications in image editing. A common scenario is to composite a portrait photo with a new background. Sample images for portrait compositing is shown in Fig. 1. To get high-quality composite images, professionals rely on image editing soft-ware to perform operations like segmentation, matting and foreground color decontamination. Although many parts of the workflow have been made relatively easier by software, it still requires a lot of expertise and manual efforts to create high-quality composited images. In this paper, we aim to fully automate the portrait image compositing process.