Enhancing the Power of OOD Detection via Sample-Aware Model Selection

In this work, we present a novel perspective on detecting out-of-distribution (OOD) samples and propose an algorithm for sample-aware model selection to enhance the effectiveness of OOD detection. Our algorithm determines, for each test input, which pre-trained models in the model zoo are capable of identifying the test input as an OOD sample. If no such models exist in the model zoo, the test input is classified as an in-distribution (ID) sample. We the-oretically demonstrate that our method maintains the true positive rate of ID samples and accurately identifies OOD samples with high probability when there are a sufficient number of diverse pre-trained models in the model zoo. Extensive experiments were conducted to validate our method, demonstrating that it leverages the complementarity among single-model detectors to consistently improve the effective-ness of OOD sample identification. Compared to baseline methods, our approach improved the relative performance by 65.40% and 37.25% on the CIFAR10 and ImageNet benchmarks, respectively.

1. Introduction

Deep neural networks have shown remarkable success in a variety of applications, but their ability to generalize ro-bustly remains a challenging issue in deep learning. While highly trained and complex deep neural networks can per-form exceptionally well on test data that is identically dis-tributed (ID) with the training data, their effectiveness in accurately predicting inputs that fall outside of the training distribution is limited. This poses a significant hurdle to the generalization capability of deep neural network models. In safety-critical applications, it is preferable to detect out-of-distribution (OOD) inputs beforehand rather than relying on the model to make potentially unreliable predictions.