Robust Commonsense Reasoning Against Noisy Labels Using Adaptive Correction

Commonsense reasoning based on knowledge graphs (KGs) is a challenging task that requires predicting complex questions over the described textual contexts and relevant knowledge about the world. However, current methods typically assume clean training scenarios with accurately labeled samples, which are often unrealistic. The training set can include mislabeled samples, and the robustness to label noises is essential for commonsense reasoning methods to be practical, but this problem remains largely unexplored. This work focuses on commonsense reasoning with mislabeled training samples and makes several technical contributions: 1) we first construct diverse augmentations from knowledge and model, and offer a simple yet effective multiple-choice alignment method to divide the training samples into clean, semi-clean, and unclean parts; 2) we design adaptive label correction methods for the semi-clean and unclean samples to exploit the supervised potential of noisy information; and 3) finally, we extensively test these methods on noisy versions of commonsense reasoning benchmarks (CommonsenseQA and OpenbookQA). Experimental results show that the proposed method can significantly enhance robustness and improve overall performance. Furthermore, the proposed method is generally applicable to multiple existing commonsense reasoning frameworks to boost their robustness. The code is available at https://github.com/xdxuyang/CR_Noisy_Labels.

I. Introduction

Commonsense reasoning [1], [2] is a challenging task, requiring complex question predictions using the described textual contexts and unstated association knowledge about the world, which has made significant progress with the development of language models (LMs) [3]. Language encoder models [4], such as BERT [5], [6] and its variants [7], [8], have outperformed on various commonsense reasoning tasks, like question answering (QA) [9], natural language inference (NLI) [10], and text generation. Recently, massive studies have demonstrated the significant role of knowledge graphs (KGs) [11], which obtain external knowledge explicitly using relationships between entities and can play in structured reasoning and query answering [12]. However, recent commonsense reasoning advances are restricted to absolutely right supervised scenarios. Consequently, they are not scalable to real-world applications [13], [14], [15] where the training labels may be incorrect, which would mislead the model to learn or memorize wrong correlations.