I. Introduction
Semantic segmentation is an important technology for many vision-based applications. Current studies [1], [2], [3], [4], [5], [6] on semantic segmentation mainly focused on designing complex segmentation networks with higher segmentation capacities on in-distribution samples whose categories are seen during training, while they paid less attention to out-of-distribution (OOD) samples, a.k.a, anomalous samples, whose categories are unknown during training. Consequently, they are incapable of identifying anomalous samples. Instead, they can only predict an anomalous sample as seen categories. This issue greatly impedes their uses in safety-critical applications such as autonomous driving in urban scenes. For example, the anomalous samples (marked by yellow boxes) in Fig. 1 are predicted as a road by a segmentation network, which may lead to accidents. To address this issue, anomaly segmentation, a task to detect and segment out unseen anomalous samples with a given pre-trained segmentation model, is attracting more and more attention.
The aim of anomaly segmentation is to detect samples whose categories are unseen during the training phase, which are called out-of-distribution (OOD) samples, using a pre-trained segmentation model. However, when faced with OOD samples, existing segmentation models can only predict them as seen categories, leading to dangerous outcomes in applications like autonomous driving. For instance, OOD samples (marked by yellow boxes) can be mistakenly recognized as in-distribution samples, such as road and people, and cause accidents. On the other hand, hard in-distribution samples (marked by red boxes) can also cause segmentation failures and be misidentified as OOD samples in anomaly segmentation.