Using BlueField-3 SmartNICs to Offload Vector Operations in Krylov Subspace Methods

Modern SmartNICs are capable of performing both computation and communication operations. In this context, past works on accelerating HPC/DL applications have manually selected some computational phases for offloading them to the SmartNICs. In this work, we identify Vector Multiply-Adds (VMA), Distributed Dot Products (DDOT), and Sparse Matrix-Vector Multiplication (Matvec) as three fundamental operations that are widely used in Krylov Subspace methods. We propose a generic scheme to automatically offload a selected set of the above operations to NVIDIA's latest BlueField-3 SmartNICs. Our proposed method works for any variant of the PCG solver algorithms. We also propose an optimization to reduce data transfer cost for offloading Matvec operation. Our proposed schemes demonstrate up to 1) 24% improvement in PCG, and Pipelined PCG algorithms on 256 processes on Intel Broadwell CPUs and 11% improvement on a system with Intel Sapphire Rapids CPU with BF3 SmartNIC using the PETSc and HYPRE solver libraries. To the best of our knowledge, this is the first work to propose a framework to efficiently offload VMA, DDOT, and Matvec operations to the DPU and show improvements on a modern CPU-based system.