Semantic Metamorphic Testing focusing on Object Rarity

Deep Learning (DL) driven image recognition systems have developed rapidly. Recent researches have applied metamorphic testing (MT) to detect image misrecognition. Most of them are based on non-semantic transformation, while semantic MTs don’t focus on rarity. Our fundamental concept is "Rare object is rarely well-recognized." In this paper, we propose a semantic rarity-based MT and its test architecture. It consists of two units: BERT-based unit and Stable-Diffusion-based unit. The former transforms a seed word into rare sentences. The latter generates corresponding rare images. Experiments are conducted and show that the semantic rarity-based MT detects more failures.