Design and Analysis of Simulation Experiments Using Three Simple Statistical Formulas

Output-data analysis is arguably the most-researched topic in the field of simulation modeling, with more than 1000 technical papers having been written. However, many of the published papers are highly mathematical in nature, making them difficult to understand for many simulation practitioners. In this tutorial, we discuss the replication and replication/deletion approaches which can address most analysis problems using three simple formulas (or expressions) from a first undergraduate statistics course. Although the replication approaches discussed above are widely used for estimating the mean of a single simulated system, we show that the same three formulas can also be used to compare any number of simulated systems, to handle multiple system performance measures simultaneously, and also to estimate performance measures such as probabilities and percentiles rather than just means. We also discuss a relatively simple graphical methodology for determining a warmup period if steady-state characteristics are of interest. Moreover, the replication approach allows one to make multiple simulation runs simultaneously using a multi-core processor or cloud computing, leading to highly-precise estimates.