Distributional Discrimination Using Kolmogorov-Smirnov Statistics and Kullback-Leibler Divergence for Gamma, Log-Normal, and Weibull Distributions

This research compares two methods of choosing a distribution to match sample data: Kullback-Leibler (KL) divergence and the Kolmogorov-Smirnoff (KS) statistic. We generate sample data from a known distribution (we used the gamma, log-normal, and Weibull distributions), find best matches to the data for each candidate distribution using maximum likelihood parameter estimation, then use KL divergence and the KS statistic to choose a best fit for the data. Using Monte-Carlo simulation, we estimate a probability of correct selection for KL divergence and the KS statistic by determining how frequently each method correctly selects the known underlying distribution. Results vary based on the data-generating distribution type, parameters, and sample size, but we find that KL divergence generally outperforms the KS statistic except in a few rare instances. This is an important result, as the two measures are not directly comparable, and are competing methods for measuring the distance between two distributions.