HSFL: Efficient and Privacy-Preserving Offloading for Split and Federated Learning in IoT Services

Distributed machine learning methods like Federated Learning (FL) and Split Learning (SL) meet the growing demands of processing large-scale datasets under privacy restrictions. Recently, FL and SL are combined in hybrid SLFL (SFL) frameworks to exploit both methods’ advantages to facilitate ubiquitous intelligence in the Internet of Things (IoT), for example, smart finance. Despite its significant impact on the performance and costs of SFL, model decomposition that splits an ML model into the client-server pair has not been sufficiently studied, especially for SFL in a large-scale dynamic IoT environment. In this paper, we propose a new SFL framework HSFL with a lightweight model decomposition method to offload a part of model training to the edge server. Specifically, we develop a method for estimating the training latency of HSFL and designed a metric for measuring privacy leakage in HSFL, based on which we formulate model decomposition in HSFL as an optimization problem with privacy protection as a constraint. Then, we transform the formulated problem into a contextual bandit problem and design an efficient algorithm to solve it. We have conducted thorough evaluations of the proposed HSFL framework through extensive experiments on a prototype testbed and a simulation platform. The experimental results validate the superiority of HSFL over the state-of-the-art benchmarks in terms of training latency, efficiency, scalability, and privacy protection.

I. Introduction

Internet of Things (IoT) generates a huge amount of data that can be exploited by Machine Learning (ML) techniques for provisioning a wide range of intelligent services such as financial services [1], smart transportation [2], [3], and smart home [4]. The end-edge-cloud architecture (Fig. 1) has become the backbone infrastructure for intelligent IoT applications [5]–[8]. In this architecture, end-user devices offload computing tasks to edge nodes and/or the cloud data center for processing. The edge nodes are typically implemented based on various network devices (e.g., access points and IoT gateways) in wireless networks, thus often having constrained computational and communication resources. Conventional ML techniques require the data generated on user/end devices to be transmitted to a central site (e.g., the cloud server), which not only compromises the privacy protection of user data but also consumes huge bandwidth in the IoT network. Therefore, the highly distributed and resource-constrained edge-cloud computing architecture calls for new efficient and privacy-preserving ML methods for smart service provisioning in IoT.