Abstract:
The importance of graphics processing units (GPUs) in accelerating HPC applications is evident by the fact that a large number of supercomputing clusters are GPU enabled....Show MoreMetadata
Abstract:
The importance of graphics processing units (GPUs) in accelerating HPC applications is evident by the fact that a large number of supercomputing clusters are GPU enabled. Many of these HPC applications use message passing interface (MPI) as their programming model. These MPI applications frequently exchange data that is noncontiguous in GPU memory. MPI provides derived datatypes (DDTs) to represent such data. Past research on DDTs mainly focused on optimizing the pack–unpack kernels. Modern HCAs are capable of gathering/scattering data from/to noncontiguous GPU memory regions. We propose a low-overhead HCA-assisted scheme to improve the performance of GPU-based noncontiguous exchanges without the GPU-based pack–unpack kernels. We show that the proposed scheme provides up to 2× benefits compared to the existing pack-based scheme at the benchmark level. Furthermore, we show up to 17% improvement with the SW4Lite application compared to other MPI libraries, such as MVAPICH2-GDR and OpenMPI+UCX.
Published in: IEEE Micro ( Volume: 43, Issue: 2, 01 March-April 2023)
Funding Agency:
The Ohio State University, Columbus, OH, USA
Kaushik Kandadi Suresh is a Ph.D. student at The Ohio State University, Columbus, OH, 43210-1277, USA, advised by Dr. D. K. Panda. His research focuses on optimizing CPU and GPU-based communication runtimes, such as MVAPICH2-X and MVAPICH2-GDR. Kandadi Suresh received a bachelor’s degree in electrical and electronics engineering from the National Institute of Technology Tiruchirappalli, Tiruchirappalli, India. He is a Gra...Show More
Kaushik Kandadi Suresh is a Ph.D. student at The Ohio State University, Columbus, OH, 43210-1277, USA, advised by Dr. D. K. Panda. His research focuses on optimizing CPU and GPU-based communication runtimes, such as MVAPICH2-X and MVAPICH2-GDR. Kandadi Suresh received a bachelor’s degree in electrical and electronics engineering from the National Institute of Technology Tiruchirappalli, Tiruchirappalli, India. He is a Gra...View more
The Ohio State University, Columbus, OH, USA
Kawthar Shafie Khorassani is a Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. Her research interests include high performance computing, and GPU communication and computation. Shafie Khorassani received a bachelor’s degree in mathematics and computer science from Wayne State University, Detroit, MI, USA. Contact her at shafiekhorassani.1@osu.e...Show More
Kawthar Shafie Khorassani is a Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. Her research interests include high performance computing, and GPU communication and computation. Shafie Khorassani received a bachelor’s degree in mathematics and computer science from Wayne State University, Detroit, MI, USA. Contact her at shafiekhorassani.1@osu.e...View more
The Ohio State University, Columbus, OH, USA
Chen Chun Chen is a third year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high performance computing and GPU communication. He is a Member of the Network Based Computing Laboratory. Contact him at chen.10252@osu.edu.
Chen Chun Chen is a third year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high performance computing and GPU communication. He is a Member of the Network Based Computing Laboratory. Contact him at chen.10252@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Bharath Ramesh is a fifth-year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance computing, architecture-aware communication, network/hardware-based offloading, and topology-aware collective algorithms. He is a Graduate Student Member of IEEE. Contact him at ramesh.113@osu.edu.
Bharath Ramesh is a fifth-year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance computing, architecture-aware communication, network/hardware-based offloading, and topology-aware collective algorithms. He is a Graduate Student Member of IEEE. Contact him at ramesh.113@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Mustafa Abduljabbar has been a research scientist at Network-Based Computing Lab, The Ohio State University, Columbus, OH, 43210-1277, USA, since May 2022. His research interests include high-performance computing systems, hardware-aware parallel optimization, parallel programming models and their interaction with other fields, such as artificial intelligence, machine learning, and numerical algorithms. Contact him at abd...Show More
Mustafa Abduljabbar has been a research scientist at Network-Based Computing Lab, The Ohio State University, Columbus, OH, 43210-1277, USA, since May 2022. His research interests include high-performance computing systems, hardware-aware parallel optimization, parallel programming models and their interaction with other fields, such as artificial intelligence, machine learning, and numerical algorithms. Contact him at abd...View more
The Ohio State University, Columbus, OH, USA
Aamir Shafi is a research scientist at The Ohio State University, Columbus, OH, 43210-1277, USA, where he is involved in the high performance big data and deep learning projects. His research interests include architecting robust communication libraries and tools for HPC systems with emphasis on machine and deep learning applications. Contact him at shafi.16@osu.edu.
Aamir Shafi is a research scientist at The Ohio State University, Columbus, OH, 43210-1277, USA, where he is involved in the high performance big data and deep learning projects. His research interests include architecting robust communication libraries and tools for HPC systems with emphasis on machine and deep learning applications. Contact him at shafi.16@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Hari Subramoni is an assistant professor in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance interconnects and protocols, parallel computer architecture, network-based computing, exascale computing, network topology aware computing, quality-of-service (QoS), power-aware local area network (LAN)-wide area network (W...Show More
Hari Subramoni is an assistant professor in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance interconnects and protocols, parallel computer architecture, network-based computing, exascale computing, network topology aware computing, quality-of-service (QoS), power-aware local area network (LAN)-wide area network (W...View more
The Ohio State University, Columbus, OH, USA
Dhabaleswar K. Panda is a professor of computer science and engineering and university distinguished scholar at The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include parallel computer architecture, high-performance networking, exascale computing, big data, deep learning, programming models, accelerators, high-performance file systems and storage, virtualization, and cloud computing. He i...Show More
Dhabaleswar K. Panda is a professor of computer science and engineering and university distinguished scholar at The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include parallel computer architecture, high-performance networking, exascale computing, big data, deep learning, programming models, accelerators, high-performance file systems and storage, virtualization, and cloud computing. He i...View more
Contents The Ohio State University, Columbus, OH, USA
Kaushik Kandadi Suresh is a Ph.D. student at The Ohio State University, Columbus, OH, 43210-1277, USA, advised by Dr. D. K. Panda. His research focuses on optimizing CPU and GPU-based communication runtimes, such as MVAPICH2-X and MVAPICH2-GDR. Kandadi Suresh received a bachelor’s degree in electrical and electronics engineering from the National Institute of Technology Tiruchirappalli, Tiruchirappalli, India. He is a Graduate Student Member of IEEE. Contact him at kandadisuresh.1@osu.edu.
Kaushik Kandadi Suresh is a Ph.D. student at The Ohio State University, Columbus, OH, 43210-1277, USA, advised by Dr. D. K. Panda. His research focuses on optimizing CPU and GPU-based communication runtimes, such as MVAPICH2-X and MVAPICH2-GDR. Kandadi Suresh received a bachelor’s degree in electrical and electronics engineering from the National Institute of Technology Tiruchirappalli, Tiruchirappalli, India. He is a Graduate Student Member of IEEE. Contact him at kandadisuresh.1@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Kawthar Shafie Khorassani is a Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. Her research interests include high performance computing, and GPU communication and computation. Shafie Khorassani received a bachelor’s degree in mathematics and computer science from Wayne State University, Detroit, MI, USA. Contact her at shafiekhorassani.1@osu.edu.
Kawthar Shafie Khorassani is a Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. Her research interests include high performance computing, and GPU communication and computation. Shafie Khorassani received a bachelor’s degree in mathematics and computer science from Wayne State University, Detroit, MI, USA. Contact her at shafiekhorassani.1@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Chen Chun Chen is a third year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high performance computing and GPU communication. He is a Member of the Network Based Computing Laboratory. Contact him at chen.10252@osu.edu.
Chen Chun Chen is a third year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high performance computing and GPU communication. He is a Member of the Network Based Computing Laboratory. Contact him at chen.10252@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Bharath Ramesh is a fifth-year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance computing, architecture-aware communication, network/hardware-based offloading, and topology-aware collective algorithms. He is a Graduate Student Member of IEEE. Contact him at ramesh.113@osu.edu.
Bharath Ramesh is a fifth-year Ph.D. student in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance computing, architecture-aware communication, network/hardware-based offloading, and topology-aware collective algorithms. He is a Graduate Student Member of IEEE. Contact him at ramesh.113@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Mustafa Abduljabbar has been a research scientist at Network-Based Computing Lab, The Ohio State University, Columbus, OH, 43210-1277, USA, since May 2022. His research interests include high-performance computing systems, hardware-aware parallel optimization, parallel programming models and their interaction with other fields, such as artificial intelligence, machine learning, and numerical algorithms. Contact him at abduljabbar.1@osu.edu.
Mustafa Abduljabbar has been a research scientist at Network-Based Computing Lab, The Ohio State University, Columbus, OH, 43210-1277, USA, since May 2022. His research interests include high-performance computing systems, hardware-aware parallel optimization, parallel programming models and their interaction with other fields, such as artificial intelligence, machine learning, and numerical algorithms. Contact him at abduljabbar.1@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Aamir Shafi is a research scientist at The Ohio State University, Columbus, OH, 43210-1277, USA, where he is involved in the high performance big data and deep learning projects. His research interests include architecting robust communication libraries and tools for HPC systems with emphasis on machine and deep learning applications. Contact him at shafi.16@osu.edu.
Aamir Shafi is a research scientist at The Ohio State University, Columbus, OH, 43210-1277, USA, where he is involved in the high performance big data and deep learning projects. His research interests include architecting robust communication libraries and tools for HPC systems with emphasis on machine and deep learning applications. Contact him at shafi.16@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Hari Subramoni is an assistant professor in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance interconnects and protocols, parallel computer architecture, network-based computing, exascale computing, network topology aware computing, quality-of-service (QoS), power-aware local area network (LAN)-wide area network (WAN) communication, fault tolerance, virtualization, deep learning, big data, and cloud computing. He is a Member of IEEE and ACM. Contact him at subramoni.1@osu.edu.
Hari Subramoni is an assistant professor in the Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include high-performance interconnects and protocols, parallel computer architecture, network-based computing, exascale computing, network topology aware computing, quality-of-service (QoS), power-aware local area network (LAN)-wide area network (WAN) communication, fault tolerance, virtualization, deep learning, big data, and cloud computing. He is a Member of IEEE and ACM. Contact him at subramoni.1@osu.edu.View more
The Ohio State University, Columbus, OH, USA
Dhabaleswar K. Panda is a professor of computer science and engineering and university distinguished scholar at The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include parallel computer architecture, high-performance networking, exascale computing, big data, deep learning, programming models, accelerators, high-performance file systems and storage, virtualization, and cloud computing. He is a Fellow of IEEE. Contact him at panda@cse.ohio-state.edu.
Dhabaleswar K. Panda is a professor of computer science and engineering and university distinguished scholar at The Ohio State University, Columbus, OH, 43210-1277, USA. His research interests include parallel computer architecture, high-performance networking, exascale computing, big data, deep learning, programming models, accelerators, high-performance file systems and storage, virtualization, and cloud computing. He is a Fellow of IEEE. Contact him at panda@cse.ohio-state.edu.View more
