Accelerating Federated Learning With Model Segmentation for Edge Networks

In the rapidly evolving landscape of distributed learning strategies, Federated Learning (FL) stands out for its features such as model training on resource-constrained edge devices and high data security. However, the growing complexity of neural network models produces two challenges such as communication bottleneck and resource under-utilization, especially in edge networks. To overcome these challenges, this paper introduces a novel framework by realizing the Parallel Communication-Computation Federated Learning Mode (P2CFed). Specifically, we design an adaptive layer-wise model segmentation strategy according to the wireless environments and computing capability of edge devices, which enables parallel training and transmission within different sub-models. In this way, parameter delivery takes place throughout the training process, thus considerably alleviating the communication overhead. Meanwhile, we also propose a joint optimization scheme with regard to the subchannel allocation, power control, and segmentation layer selection, which is then transformed into an iteration search process for obtaining optimal results. We have conducted extensive simulations to validate the effectiveness of P2CFed when compared with state-of-the-art benchmarks in terms of communication overhead and resource utilization. It also unveils that P2CFed brings a faster convergence rate and smaller training delay compared to traditional FL approaches.

I. Introduction

In conventional edge networks, there exists massive data produced at edge terminals deployed around the world, which presents a huge challenge in the management of these large amounts of data such as how to accomplish data transmission and processing as soon as possible while maintaining the data security [1]. To overcome this challenge, the Federated Learning (FL) model has been introduced [2], which is well-known for its two distinguishing features. First, as demonstrated in Fig. 1, unlike the traditional machine learning model requiring gathering data from edge terminals and then introducing considerable communication overhead [3], [4], [5], Local training can be performed in the FL framework in order to make full use of the computing capability in edge devices. Secondly, only the trained model parameters, instead of the original data, should be transmitted from edge devices to the center server for model aggregation thus considerably reducing the amount of transmitted data. Fig. 1.

The edge networks.