Computing with heterogeneous parallel machines: advantages and challenges

Discusses the advantages of computing with heterogeneous parallel machines, and examines the research challenges for automating the use of such systems. One type of heterogeneous computing system is a mixed-mode machine, where a single machine can operate in different modes of parallelism. Another is a mixed-machine system, where a suite of different kinds of parallel machines are interconnected by high-speed links. To exploit such systems, a task must be decomposed into subtasks, where each subtask is computationally homogeneous. The subtasks are then assigned to and executed with the machines (or modes) that will result in a minimal overall execution time. Typically, users must specify this decomposition and assignment. One long-term pursuit in heterogeneous computing is to do this automatically. An overview of a conceptual model of what this involves is given. As an example of the research in this area, a genetic-algorithm-based approach to the subtask assignment and scheduling problem is explored. Open problems in heterogeneous computing are described.