When Computing Power Network Meets Distributed Machine Learning: An Efficient Federated Split Learning Framework

In this paper, we advocate CPN-FedSL, a novel and flexible Federated Split Learning (FedSL) framework over Computing Power Network (CPN). We build a dedicated model to capture the basic settings and learning characteristics (e.g., training flow, latency and convergence). Based on this model, we introduce Resource Usage Effectiveness (RUE), a novel performance metric integrating training utility with system cost, and formulate a multivariate scheduling problem that maximizes RUE by comprehensively taking client admission, model partition, server selection, routing and bandwidth allocation into account (i.e., mixed-integer fractional programming). We design Refinery, an efficient approach that first linearizes the fractional objective and non-convex constraints, and then solves the transformed problem via a greedy based rounding algorithm in multiple iterations. Extensive evaluations corroborate that CPN-FedSL is superior to the standard and state-of-the-art learning frameworks (e.g., FedAvg and SplitFed), and besides Refinery is lightweight and significantly outperforms its variants and de facto heuristic methods under a variety of settings.

I. Introduction

The significant trend of network and computing convergence in the upcoming Network 2030 [1], [2] inspires a new concept of Computing Power Network (CPN), also known as Compute First Networking (CFN) [3], [4]. CPN aims to connect diverse computing power in the cloud, on the edge, and across devices to implement on-demand resource scheduling. Recently, ITU-T has launched the first CPN standard [5] to facilitate its proactive development, and many organizations have incorporated CPN into their future research, such as “Eastern Data and Western Computing” in China and “Sky Computing” advocated by UC Berkeley [6]. It can be foreseen that CPN will be a key scenario to boost many emerging services (e.g., extended reality and big data analytics).