Sparse Matrix-Vector Multiplication with Reduced-Precision Memory Accessor

Mixed-precision computation, which uses multiple different precision in a single code, is being studied to increase computational speed and energy efficiency. It typically uses the IEEE 754-2008 floating-point formats; however, for further fine-grained precision tuning, it is possible to consider using custom data representations, for example, with lower mantissa precision than the IEEE formats. Sparse iterative solvers are examples in which the use of such reduced-precision formats is effective. In this study, we discuss the implementation and performance of sparse matrix-vector multiplication (SpMV), which is the kernel of sparse iterative solvers, with reduced-precision floating-point formats on general-purpose processors. The reduced-precision scheme we adopt truncates the mantissa bits of the IEEE formats in 8-bit steps (e.g., it allows the creation of 16-/24-/32-/40-/48-/56-bit formats based on binary64). It acts as a so-called memory accessor: the reduced-precision format is used only for data representation in memory, and arithmetic operations are performed in the IEEE format on the register. It can be easily introduced without significant code changes. With this scheme, we implement two types of reduced-precision SpMVs: the compressed sparse row format, which is a classical sparse format, and the adaptive multi-level blocking format, which is an index-compressed format based on the Sell-C-σ format. We evaluate performance on four different architectures, including CPUs and GPUs (AMD Zen3, Intel Ice Lake, NVIDIA Ampere, and AMD CDNA). As a result, we demonstrate good scalability for the bit length of the format, as long as the problem is well out-of-cache and performance is memory-bandwidth-bound.