A Joint Communication and Learning Framework for Hierarchical Split Federated Learning

In contrast to methods relying on a centralized training, emerging Internet of Things (IoT) applications can employ federated learning (FL) to train a variety of models for performance improvement and improved privacy preservation. FL calls for the distributed training of local models at end-devices, which uses a lot of processing power (i.e., CPU cycles/sec). Most end-devices have computing power limitations, such as IoT temperature sensors. One solution for this problem is split FL. However, split FL has its problems, including a single point of failure, issues with fairness, and a poor convergence rate. We provide a novel framework, called hierarchical split FL (HSFL), to overcome these issues. On grouping, our HSFL framework is built. Partial models are constructed within each group at the devices, with the remaining work done at the edge servers. Each group then performs local aggregation at the edge following the computation of local models. End devices are given access to such an edge aggregated model so they can update their models. For each group, a unique edge aggregated HSFL model is produced by this procedure after a set number of rounds. Shared among edge servers, these edge aggregated HSFL models are then aggregated to produce a global model. Additionally, we propose an optimization problem that takes into account the relative local accuracy (RLA) of devices, transmission latency, transmission energy, and edge servers’ compute latency in order to reduce the cost of HSFL. The formulated problem is a mixed-integer nonlinear programming (MINLP) problem and cannot be solved easily. To tackle this challenge, we perform decomposition of the formulated problem to yield subproblems. These subproblems are edge computing resource allocation problem and joint RLA minimization, wireless resource allocation, task offloading, and transmit power allocation subproblem. Due to the convex nature of edge computing, resource allocation is done so utilizing a convex optimizer, as opposed to a block successive upper-bound minimization (BSUM)-based approach for joint RLA minimization, resource allocation, job offloading, and transmit power allocation. Finally, we present the performance evaluation findings for the proposed HSFL scheme.

I. Introduction

Applications for the Internet of Things (IoT) are designed to serve a large number of users by satisfying their different needs [1], [2], [3], [4], [5], [6]. IoT solutions must be carefully designed in order to meet these needs. Effectively modeling of IoT network functions can help us optimize the performance of IoT systems. These techniques can be based on a variety of theories, including optimization theory, game theory, graph theory, and heuristics. However, some IoT issues seem to be challenging to accurately model using the aforementioned techniques [7], [8], [9], [10]. One can utilize machine learning (ML) to get around this restriction. Generally, centralized ML relies on moving data from end devices to a single location for training, and thus it suffers from privacy leaks. Federated learning (FL), which does not need transferring data from devices to a centralized server for training, can be used to address the privacy leakage problem associated with centralized ML [11], [12], [13]. Devices in FL learn their local models and send them to a server at the edge or in the cloud where they are aggregated to produce a global model. The FL has some challenges, including resource optimization, single point of failure (SPF), incentive design, and learning algorithm design, despite the fact that it can maintain privacy more effectively than centralized ML [14], [15], [16]. Additionally, a local learning model can also be unable to be trained within the allotted time on devices with insufficient computational power.