Quality assurance study with mismatched data in sentiment analysis

Considering mismatched data have harmful influence on the quality assurance in sentiment analysis, therefore this paper proposed an effective method to detect these mismatches. This study considered data adequacy and model’s confidence together, and proposed to use the surprise adequacy metric for mismatch detection. Experiments were implemented on Amazon.com review data. Performances of mismatched data detection and model retraining were evaluated. The proposed method using Mahalanobis-distance-based surprise adequacy was verified feasible and effective to detect mismatched data in the studied dataset. Moreover, after mismatch detection, retrained model was illustrated useful to improve AI model’s quality.