Loading [MathJax]/extensions/MathZoom.js
Strategies for Fast I/O Throughput in Large-Scale Climate Modeling Applications | IEEE Conference Publication | IEEE Xplore

Strategies for Fast I/O Throughput in Large-Scale Climate Modeling Applications


Abstract:

Large-scale HPC applications are highly data-intensive with significant times spent in I/O operations. Many large-scale scientific applications do not adequately optimize...Show More

Abstract:

Large-scale HPC applications are highly data-intensive with significant times spent in I/O operations. Many large-scale scientific applications do not adequately optimize the I/O operations, leading to overall poor performance. In this work, we have developed two main strategies for providing fast I/O throughput for an important climate modeling application, namely, Regional Ocean Modeling System (ROMS) that uses NetCDF for I/O operations. The strategies include load balancing the I/O operations and selective writing of data. We have also implemented file striping to improve I/O performance. Our experiments with up to 1440 processor cores and 5 days of simulations showed that our load balancing strategy resulted in about 27 % decrease in execution times over the default executions, our selective writing strategy resulted in a further decrease of about 30 % and the optimized file striping resulted in a further decrease of about 12 % in execution times. All the strategies combined together improved the overall performance of the application by about 70 %.
Date of Conference: 18-21 December 2023
Date Added to IEEE Xplore: 05 April 2024
ISBN Information:

ISSN Information:

Conference Location: Goa, India
Qualcomm, Hyderabad, India
Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, India
Centre for Atmospheric and Oceanic Sciences, Indian Institute of Science, Bangalore, India

I. Introduction

ROMS [1] is an ocean model widely used by the scientific community for a wide range of climate applications. It can work standalone as well as it can be coupled to atmospheric and/or wave models. It is built based on the Earth System Modeling Framework (ESMF) [2] which provides high per-formance and flexibility for coupling climate and related other scientific applications. The format of the input and output data of the model are Network Common Data Form (NetCDF) [3] which helps to interchange the data in a user-friendly way.

Qualcomm, Hyderabad, India
Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, India
Centre for Atmospheric and Oceanic Sciences, Indian Institute of Science, Bangalore, India

References

References is not available for this document.