Mask-Guided Matting in the Wild

Mask-guided matting has shown great practicality compared to traditional trimap-based methods. The mask-guided approach takes an easily-obtainable coarse mask as guidance and produces an accurate alpha matte. To extend the success toward practical usage, we tackle mask-guided matting in the wild, which covers a wide range of categories in their complex context robustly. To this end, we propose a simple yet effective learning framework based on two core insights: 1) learning a generalized matting model that can better understand the given mask guidance and 2) leveraging weak supervision datasets (e.g., instance segmentation dataset) to alleviate the limited diversity and scale of existing matting datasets. Extensive experimental results on multiple benchmarks, consisting of a newly proposed synthetic benchmark (Composition-Wild) and existing natural datasets, demonstrate the superiority of the proposed method. Moreover, we provide appealing results on new practical applications (e.g., panoptic matting and mask-guided video matting), showing the great generality and potential of our model.

1. Introduction

Image matting aims to predict the opacity of objects, which enables precise separation from surrounding backgrounds. Due to the ill-posed nature of the task, many works [7], [13], [21], [27], [30], [48] have improved matting performance by relying on the manual guidance of a trimap. However, pixel-level annotation of fore-ground/background/unknown is extremely burdensome, restricting its usage in many practical applications such as image/video editing and film production. Recently, many efficient alternatives for user guidance have been proposed, including trimap-free [15], [32], additional background images [22], [34], scribble [43], and the user clicks [45].