Experimental Corroboration of Trained Classification Performance Predictions

A recent paper established asymptotic expressions for classification performance appropriate when the decision is based on training data. These expressions are asymptotically rigorous, and aspirational in the sense that they show what could be done with the best use made of available tools; and they are exponential with rate that - remarkably - scales with the number of relevant data in the test observation (e.g., number of independent observations, or the size of a target to be detected within an image, irrespective of the total image size). The paper also showed a close approximation of this performance that seems to give high accuracy even in non-asymptotic situations. In this paper we briefly present these results, but the main contribution is to demonstrate their validity and general applicability in the standard MNIST digit classification set.