HBalancer: A machine learning based load balancer in real time CPU-GPU heterogeneous systems

Graphical Processing Units (GPUs) are increasingly being incorporated to High-performance computing (HPC) systems alongside Central Processing Units (CPUs). As a result, HPC systems turned into heterogeneous systems.The unequal distribution of computing loads to the devices of a heterogeneous system is a major issue. It is caused by the variation of computing power between these devices. Balancing the load of such systems is a very complex task.In this context, we provide HBalancer, a CPU-GPU heterogeneous system resource manager. HBalancer distributes computing loads on the devices at runtime in a manner that minimizes the system imbalance. We define a new metric to estimate the system imbalance using execution time prediction. We provide mathematical formulas to calculate it.Experiments were conducted on a CPU-GPU heterogeneous system. We used multiple OpenCL applications as workload. Results show that HBalancer outperforms the Device Suitability and Round Robin approaches in load balance and execution time.