Heroes: Lightweight Federated Learning with Neural Composition and Adaptive Local Update in Heterogeneous Edge Networks

Federated Learning (FL) enables distributed clients to collaboratively train models without exposing their private data. However, it is difficult to implement efficient FL due to limited resources. Most existing works compress the transmitted gradients or prune the global model to reduce the resource cost, but leave the compressed or pruned parameters under-optimized, which degrades the training performance. To address this issue, the neural composition technique constructs size-adjustable models by composing low-rank tensors, allowing every parameter in the global model to learn the knowledge from all clients. Nevertheless, some tensors can only be optimized by a small fraction of clients, thus the global model may get insufficient training, leading to a long completion time, especially in heterogeneous edge scenarios. To this end, we enhance the neural composition technique, enabling all parameters to be fully trained. Further, we propose a lightweight FL framework, called Heroes, with enhanced neural composition and adaptive local update. A greedy-based algorithm is designed to adaptively assign the proper tensors and local update frequencies for participating clients according to their heterogeneous capabilities and resource budgets. Extensive experiments demonstrate that Heroes can reduce traffic consumption by about 72.05% and provide up to 2.97× speedup compared to the baselines.

I. Introduction

In traditional machine learning approaches, data is gathered from various sources and transmitted to a central server for model training. However, in edge computing (EC), where data is generated and processed at the edge clients, this centralized approach becomes infeasible due to privacy concerns [1]. Federated Learning (FL) [2] is an emerging paradigm that enables collaborative model training across distributed clients, bringing the power of machine learning to EC [3]. As the field of FL continues to evolve, it carries significant potential to advance edge computing capabilities and facilitate intelligent applications across diverse domains, such as healthcare, smart cities and autonomous vehicles [4].