Robust Practical Binary Optimization at Run-time using LLVM

In High Performance Computing (HPC) the performance of applications is paramount, which is has led to a wide body of work on optimizing compilers. However, compilers alone are naturally limited in their potential, as they cannot exploit run-time information to specialize generated code. For this reason, run-time components to perform such specializations are gaining traction. In particular, application or use-case driven specializations have a high potential for optimizations, as they can benefit from explicit guidance from the application or the user, which allows for targeted changes incorporating specific knowledge. Because full compilers and sources are usually unavailable during computation, specializations have to be done at machine code level. However, existing libraries for binary specialization face structural limitations of separating optimization and code generation, in addition to numerous implementation gaps. In this paper, we describe BinOpt, a novel and robust library for performing application-driven binary optimization and specialization using LLVM. A machine code function is lifted directly to LLVM-IR, optimized in LLVM while making use of application-specified information, used to generate a new specialization for a function, and integrated back to the original code. We apply this technique to existing optimized code and show that significant performance improvements can be observed with only a small optimization time overhead.