Measuring Predictability of Nvidia's GPU Schedulers: Application to the Summation Problem

GPU's are massively multicore architectures managing several thousands of concurrent threads. This concurrency, maintained through several schedulers, is necessary to keep high performance but negatively impact predictability. The lack of predictability is not a problem for most of data parallel applications written in CUDA and therefore hasn't been widely studied. However for some others, such as the summation of floating-point numbers, this may be problematic as it can lead to deadlock situation. In this work, we first propose measures of predictability as well as CUDA tests to estimate this measure regarding warp and block scheduler for architectures from G80 to GK104. Then, we evaluate how to impact this measure and apply those results to the atomic addition of floating-point numbers and show how to make this operation predictable.