Performance Analysis of Compressed Batch Matrix Operations on Small Matrices

Dense matrix computations with very small matrices present unique challenges for performance optimization and occupy and important space in many HPC computations including PDE solvers, machine learning algorithms, and Kalman filters. Using batch computation can improve their performance significantly and compressed batch (also called block-interleaved) data structures can further improve performance. In this paper we present a detailed study of how compressed batch computations use HPC hardware and how they can be most effectively tuned for cache performance.