MR-tree: A Parametric Family of Multi-Rail Fat-tree

Although using Multi-Rail network is a popular choice for many HPC systems to overcome bandwidth limitations, the influence of connection modes between multi-port nodes and switching network is not well understood so far. This work provides a detailed analysis of different node connection modes in Multi-Rail Fat-tree. To gain a deep understanding of relevant issues, we propose a new parametric family of Multi-Rail Fat-tree called MR-tree, which could cover all balanced node connection modes for Multi-Rail Fat-tree. We evaluate different node connection modes by theoretical analysis and flit level simulation. We show that there are great differences across various node connection modes. Among the key differences are the following: cost, average shortest path length, fault-tolerance, and path diversity. In addition, our simulation results reveal that the performance gap for Multi-Rail Fat-trees with different node connection modes may vary significantly depending on the oversubscribed conditions. The above results leave open the possibility that optimization of node connection mode can yield better results for Multi-Rail Fat-tree.