Mask Guided Matting via Progressive Refinement Network

We propose Mask Guided (MG) Matting, a robust matting framework that takes a general coarse mask as guidance. MG Matting leverages a network (PRN) design which encourages the matting model to provide self-guidance to progressively refine the uncertain regions through the decoding process. A series of guidance mask perturbation operations are also introduced in the training to further enhance its robustness to external guidance. We show that PRN can generalize to unseen types of guidance masks such as trimap and low-quality alpha matte, making it suitable for various application pipelines. In addition, we revisit the foreground color prediction problem for matting and propose a surprisingly simple improvement to address the dataset issue. Evaluation on real and synthetic benchmarks shows that MG Matting achieves state-of-the-art performance using various types of guidance inputs. Code and models are available at https://github.com/yucornetto/MGMatting.

1. Introduction

Image matting is a fundamental computer vision problem which aims to predict an alpha matte to precisely cut out an image region. It has many applications in image and video editing [39], [41], [21]. Most previous matting methods require a well-annotated trimap as an auxiliary guidance input [39], which explicitly defines the regions of foreground and background as well as the unknown part for the matting methods to solve. Although such annotation makes the problem more tractable, it can be quite burdensome for users and limits the usefulness of these methods in many non-interactive applications.