Delving into High-Quality Synthetic Face Occlusion Segmentation Datasets

This paper performs comprehensive analysis on datasets for occlusion-aware face segmentation, a task that is crucial for many downstream applications. The collection and annotation of such datasets are time-consuming and labor-intensive. Although some efforts have been made in synthetic data generation, the naturalistic aspect of data remains less explored. In our study, we propose two occlusion generation techniques, Naturalistic Occlusion Generation (NatOcc), for producing high-quality naturalistic synthetic occluded faces; and Random Occlusion Generation (RandOcc), a more general synthetic occluded data generation method (Figure 1). We empirically show the effectiveness and robustness of both methods, even for unseen occlusions. To facilitate model evaluation, we present two high-resolution real-world occluded face datasets with finegrained annotations, RealOcc and RealOcc-Wild, featuring both careful alignment preprocessing and an in-the-wild setting for robustness test. We further conduct a comprehensive analysis on a newly introduced segmentation benchmark, offering insights for future exploration. Our code and dataset are available at https://github.com/kennyvoo/face-occlusion-generation.

1. Introduction

This paper explores the problem of occlusion-aware face segmentation, which involves extracting face pixels from an occluded face where occlusions are treated as the background class. Face or occlusion semantic segmentation is extensively used in face-related tasks, such as face recognition [31], face-swapping [25], [27], and facial reconstruction [32], [38]. Occlusions on faces remained less explored in the past. In reality, human faces are likely to be partially occluded by hands or wearable objects such as sunglasses or face masks. These occlusions would often degrade the performance of the face-related tasks. Therefore, occlusion-aware face segmentation is worth studying for many practical applications [18], [25], [38].