Debiased Device Sampling for Federated Edge Learning in Wireless Networks

As a privacy-preserved distributed machine learning paradigm, federated edge learning (FEL) was designed to absorb knowledge from user devices to construct intelligent services without transmitting raw data. However, this paradigm depends on the local training and model parameter transmission of user devices, therefore the computing power, storage capacity and network resources of the devices become the key factors to achieve energy well-budgeted and timely message transmission FEL. While in the wireless networks, those resources for devices are normally heterogeneous or limited. This paper aims to offer tangible solutions for optimal convergence and Quality of Service (QoS) assurance of FEL in wireless networks. First, we define a mathematical model for energy-efficient message transmission of FEL and formulate an optimization problem involving device sampling and resource allocation to attain optimal training convergence within energy and time constraints. Second, we theoretically analyze the impact of limited resources on sampling strategies and training convergence, thus simplifying the optimization problem for solvability. Third, we introduce an iterative heuristic algorithm that utilizes available resources to reduce client sampling bias. Extensive experiments show that our method can effectively obtain the debiased sampling strategy, and outperforms similar methods by minimizing device disconnection due to energy use and enhancing model convergence and performance.

I. Introduction

Due to the popularity of various user terminals such as wearable medical devices, industrial intelligent sensors and smart vehicles, the new generation of intelligent applications such as smart healthcare [1], industry [2] and transportation [3] have become the focus of research and development in the fields of communication, computer and artificial intelligence (AI). At the same time, user-side data also ushered in explosive growth, which promoted the rapid development of data-driven AI technologies such as deep learning (DL) [4]. Building this kind of intelligent application needs to collect a large number of user data to a single spot, and through DL, these data are transformed into neural network models with image recognition, word processing and voice recognition functions [5], [6], [7]. Although this centralized training paradigm has performance advantages, it also brings potential risks on privacy, data property rights and single point of failure, which make it difficult to guarantee the reliability of the system. In addition to model performance, some metrics about quality of service (QoS), such as communication latency and energy consumption, are also important evaluation indicators of application competence [8].