Category-Adaptive Label Discovery and Noise Rejection for Multi-Label Recognition With Partial Positive Labels

As a cost-effective alternative to standard multi-label learning, the multi-label image recognition with partial positive labels (MLR-PPL) task attracts increasing attention, in which merely a portion of positive labels are given while the rest of positive labels and all negative labels are missing. To facilitate this task, we propose a novel framework that leverages semantic correlation among different images in a category-adaptive manner to complement unknown labels accurately. Specifically, the proposed framework consists of two complementary modules. 1) A category-adaptive label discovery (CALD) module is designed to measure the semantic similarity between positive samples and then complement unknown labels with high similarities. 2) A category-adaptive noise rejection (CANR) module is designed to compute the sample weights based on semantic similarities from different samples and discard noisy labels with low weights. Due to the various degrees of confidence calibration among different categories, searching appropriate thresholds for each category in the proposed framework is highly time-consuming. To avoid such a resource-intensive manual tuning, we introduce a category-adaptive threshold updating algorithm that introduces the category-specific positive and negative similarity to adjust the threshold adaptively. Extensive experiments on various benchmarks show that the proposed framework performs better than current state-of-the-art algorithms.

I. Introduction

In Recent years, extensive efforts [1], [2], [3], [4], [5], [6] have been dedicated to the task of multi-label image recognition (MLR), as it offers numerous benefits to various applications ranging from video parsing [7], [8], [9] and scene recognition [10], [11], [12] to human activity analysis [13], [14], [15] and facial analysis [16], [17], [18]. Despite impressive progress, existing approaches presume that each training sample contains complete labels, which require significant staffing and resources to collect large-scale multi-label datasets for training MLR models. Such a collection process is time-consuming, expensive, and impractical, especially when the number of target categories is significant. In light of this, the community's attention has shifted towards weakly supervised multi-label image recognition (WSMLR) [19], [20], [21], which aims to reduce the dependence on complete annotations of multi-label datasets. Among different settings of WSMLR, multi-label recognition with partial labels (MLR-PL) [22], [23], [24], [25] attracts growing attention since it allows training MLR models with incomplete labels per image.