Automatic Model Evaluation using Feature Importance Patterns on Unlabeled Data

Recent studies have shown that the estimated efficacy of a classification model in a specific training set can be very different from the same model efficacy after deployment, or when the model is evaluated in a dataset with a different distribution from the training set. This situation is known as distribution shift or dataset shift, and an emerging strategy for this problem is to estimate the efficacy of the classification model in unlabeled data with unknown distribution (i.e., aka AutoEval approaches). Most of the recent works study how distribution shift affects Deep Learning Models applied to Computer Vision tasks (i.e., unstructured data). However, distribution shift also can affect the efficacy of a model on tabular/structured data. In this work, we proposed and analyzed AutoEval approaches on tabular data. We proposed an AutoEval method based on the use of feature importance, which are typically used as model explanations, to detect patterns of correct and incorrect classifications that can be used to estimate model efficacy. We conducted experiments using six real-world datasets related to three different subjects. Our results indicated that the proposed method outperforms all baselines, with a reduction up to 89% in the gap between estimated and real efficacy, in comparison with standard 10-fold cross-validation (CV) error estimation. Besides, we evaluated our AutoEval approaches as indicators to model selection in the feature selection task. In this task, compared to CV, our proposed method achieved gains up to 35%.