Tuning Parallel Data Compression and I/O for Large-scale Earthquake Simulation

Scientific applications, such as those simulating earthquakes, the origins of universe, etc., often produce massive amounts of data as high-performance computing (HPC) systems are moving toward exascale. The ever-increasing volumes of data are posing challenges for scientists to store, share, analyze, and visualize. Compression algorithms have become a crucial component for data management in scientific workflows. Data reduction enables simulations to output more data without worrying about exceeding storage quotas, and could capture more insights in the simulation. However, due to the complexity and poor performance of I/O and compression libraries as well as parallel file systems, the overall compression and I/O performance varies significantly. In this paper, we explore tuning parallel compression of data produced by a large-scale earthquake simulation. We show that our strategies achieve up to 13X performance improvement and a compression ratio of up to 251.

I. Introduction

Large-scale scientific simulation that use HPC systems are producing and analyzing data that amounts to hundreds of terabytes or even several petabytes. The volume of data is projected to increase even further with the deployment of exascale supercomputers, such as Frontier [1] at Oak Ridge Leadership Computing Facility (OLCF) and Aurora [2] at Argonne Leadership Computing Facility (ALCF).