Auto-tuning Parameter Choices in HPC Applications using Bayesian Optimization

High performance computing applications, runtimes, and platforms are becoming more configurable to enable applications to obtain better performance. As a result, users are increasingly presented with a multitude of options to configure application-specific as well as platform-level parameters. The combined effect of different parameter choices on application performance is difficult to predict, and an exhaustive evaluation of this combinatorial parameter space is practically infeasible. One approach to parameter selection is a user-guided exploration of a part of the space. However, such an ad hoc exploration of the parameter space can result in suboptimal choices. Therefore, an automatic approach that can efficiently explore the parameter space is needed. In this paper, we propose HiPerBOt, a Bayesian optimization based configuration selection framework to identify application and platform-level parameters that result in high performing configurations. We demonstrate the effectiveness of HiPerBOt in tuning parameters that include compiler flags, runtime settings, and application-level options for several parallel codes, including, Kripke, Hypre, LULESH, and OpenAtom.

I. Introduction

High performance computing (HPC) applications, runtimes, and platforms have become highly configurable. The high configurability enables applications to extract maximal performance. Unfortunately, for users, the same configurability has led to an increasingly large combinatorial search space of application-specific as well as platform-level parameters. These configurable parameters can be compiler flags, runtime system settings, application-specific parameters, hardware-level options etc. A particular choice of values for all the parameters, often called a configuration, can significantly impact various application performance metrics, such as execution time, power consumption, and system throughput among other things.