Bug Characteristics in Probabilistic Programming Systems: A Comprehensive Study

Probabilistic programming systems allow developers to model random phenomena and perform reasoning about the model efficiently. As the number and adoption of probabilistic programming systems are growing significantly, the reliability of such systems is also becoming increasingly important. It is thus crucial to analyze existing bugs in similar systems to ensure the reliability of these systems. This paper conducts an empirical study to investigate bugs and their features on real probabilistic programming systems. We select three popular probabilistic programming projects on GitHub, namely Edward, Pyro, and PyMC3, and then manually analyze a sample set of bugs to label them in different categories. The results show that 29% of the bugs belong to the algorithm/method category, 22.33 % belong to the internal interface category, and 13.67 % of the bugs belong to the external interface category. We also report the relationship between bug categories and bug-fixing duration, the effort needed to fix the bugs, and bug impacts on files.