T2HT : Traffic-Driven Machine Learning Based Hierarchical Topology Generation Model

In high-performance computing (HPC) and distributed computing area, network performance greatly influenced application efficiency. However, due to the diversity of traffic patterns, the traditional network with fixed topology may achieve good performance under some applications, while performs poorly under other forms. Network reconfiguration technologies which can change the topology dynamically have been developed to obtain a balanced performance for different traffic patterns. Nonetheless, selecting an appropriate network topology from the wide variety of options remains difficult due to the complexity of analyzing traffic alongside topology performance characteristics. Traditional research focused on congestion estimation and specific parameter adjustment without reconfiguring the global topology. In this paper, we propose a generic Traffic to Hierarchical Topology (T2HT) method to analyze traffic patterns and choose an appropriate network configuration for the given traffic T2HT makes use of actual traffic data with a hierarchical model to predict network performance with a given topology and uses a machine learning (ML) algorithm to score the better options in order to determine the best topology. We performed 8000 simulations of dataset-topology combinations to verify the feasibility of the model. Our results show that T2HT achieved marked improvements with its recommendations, making it feasible for use in hierarchical network design. Under the DOE testbed, the throughput of the topology generated by T2HT can reach above 90% of theoretical limit(full connection), and the latency is improved by about 24.6% compared to typical topology 3D Torus with the same physical restrictions.