SSN: Soft Shadow Network for Image Compositing

We introduce an interactive Soft Shadow Network (SSN) to generates controllable soft shadows for image compositing. SSN takes a 2D object mask as input and thus is agnostic to image types such as painting and vector art. An environment light map is used to control the shadow’s characteristics, such as angle and softness. SSN employs an Ambient Occlusion Prediction module to predict an intermediate ambient occlusion map, which can be further refined by the user to provides geometric cues to modulate the shadow generation. To train our model, we design an efficient pipeline to produce diverse soft shadow training data using 3D object models. In addition, we propose an inverse shadow map representation to improve model training. We demonstrate that our model produces realistic soft shadows in real-time. Our user studies show that the generated shadows are often indistinguishable from shadows calculated by a physics-based renderer and users can easily use SSN through an interactive application to generate specific shadow effects in minutes.

1. Introduction

Our Soft Shadow Network (SSN) produces convincing soft shadows given an object cutout mask and a user-specified environment lighting map. (a) shows the object cutouts for this demo, including different object categories and image types, e.g. sketch, picture, vector arts. In (b), we show the soft shadow effects generated by our SSN. The changing lighting map used for these examples is shown at the corner of (b). The generated shadows have realistic shade details near the object-ground contact points and enhance image compositing 3D effect. More animated results can be found on our project page (https://shengcn.github.io/SSN).