Runtime Adaptation for Autonomic Heterogeneous Computing

Heterogeneity is increasing at all levels of computing, certainly with the rise in general purpose computing with GPUs in everything from phones to supercomputers. More quietly it is increasing with the rise of NUMA systems, hierarchical caching, OS noise, and a myriad of other factors. As heterogeneity becomes a fact of life at every level of computing, efficiently managing heterogeneous compute resources is becoming a critical task. The focus of my dissertation is developing methods and systems to allow software to adapt to the heterogeneous hardware it finds at runtime. The goal is to make the complex functions of heterogeneous computing autonomic, handling load balancing, memory coherence and other performance critical factors in the runtime. The investigation began by studying heterogeneity caused by system topology and resource contention in MPI applications. Since then the focus has shifted to work-sharing across CPU and GPU resources for accelerated OpenMP, and automatically managing the hardware capability imbalances between these resources. Moving forward, I propose to produce a system extending upon both previous approaches to offer work-sharing, topology aware affinity management, as well as novel automated memory transformations to reduce communication and increase memory access efficiency.