Accelerating Federated Learning With Cluster Construction and Hierarchical Aggregation

Federated learning (FL) has emerged in edge computing to address the limited bandwidth and privacy concerns of traditional cloud-based training. However, the existing FL mechanisms may lead to a long training time and consume massive communication resources. In this paper, we propose an efficient FL mechanism, namely FedCH, to accelerate FL in heterogeneous edge computing. Different from existing works which adopt the pre-defined system architecture and train models in a synchronous or asynchronous manner, FedCH will construct a special cluster topology and perform hierarchical aggregation for training. Specifically, FedCH arranges all clients into multiple clusters based on their heterogeneous training capacities. The clients in one cluster synchronously forward their local updates to the cluster header for aggregation, while all cluster headers take the asynchronous method for global aggregation. Our analysis shows that the convergence bound depends on the number of clusters and the training epochs. We propose efficient algorithms to determine the optimal number of clusters with resource budgets and then construct the cluster topology to address the client heterogeneity. Extensive experiments on both physical platform and simulated environment show that FedCH reduces the completion time by 49.5-79.5% and the network traffic by 57.4-80.8%, compared with the existing FL mechanisms.

1 Introduction

In the era of big data, billions of Internet of Thing devices and smartphones around the world produce a significant amount of data per second [1]. Therefore, the traditional way of uploading those data to the remote cloud for processing will encounter many issues, including privacy leakage, network congestion, and high transmission delay [2]. Since data are generated at the network edge, mobile edge computing (MEC) is a natural alternative[3], [4], which uses the computing and storage resources of devices to perform data processing close to the data generators. According to Cisco's survey, most IoT-created data will be stored, processed, and analyzed close to or at the network edge[5]. With more data and advanced applications (e.g., autonomous driving, virtual reality, and image classification), machine learning (ML) tasks will be a dominant workload in MEC [6]. To alleviate the network bandwidth burden and avoid the privacy leakage, FL becomes an efficient solution to analyze and process the distributed data on end devices for those ML tasks [7], [8].