Analyzing the impact of floating-point precision adaptation in iterative programs

The amount of floating-point computations in numerical programs is ever increasing. In order to improve their performance, the current trend is to carefully adapt the precision of some floating-point operations, to take advantage of new features of modern architectures. Such adaptation process requires tools to evaluate the impact of these transformations on the accuracy of the output. This article presents an analysis tool to help developers in evaluating the impact of adapting the precision in floating-point programs, with a particular focus on iterative programs. This tool is implemented in the LLVM compilation framework, and it includes two main modules: fp2mp that instruments floating-point programs with multiple-precision computations, and a loop splitter that splits the iteration space of a loop into several subspaces. We illustrate the interest of this tool on various examples, and particularly how it enables to evaluate automatically the impact of adapting the precision of floating-point operations in loops by considering different precisions over the iteration space.

I. Introduction

Numerical applications are ever more complex, and the amount of floating-point computations is ever increasing. To design such applications, developers must choose between different formats. Indeed the IEEE 754–2019 standard defines four floating-point formats, i.e. binary16, binary32, binary64, and binary128 [1]. Various non IEEE formats also exist, like bfloat16 or Posit, for example. Yet, most of these applications use the binary64 floating-point format, that is the most accurate floating-point format directly available in hardware on most modern architectures, and the others remain underused. This choice, very often made independently of the expected result accuracy, leads to an underuse of the architecture features, and consequently to a possible loss of performance.