ASMAFL: Adaptive Staleness-Aware Momentum Asynchronous Federated Learning in Edge Computing

Compared with synchronous federated learning (FL), asynchronous FL (AFL) has attracted more and more attention in edge computing (EC) fields because of its strong adaptability to heterogeneous application scenarios. However, the non-independent and identically distributed (Non-IID) data across devices and the staleness-aware estimation of unreliable wireless connections and limited edge resources make it much more difficult to achieve better AFL-related applications. To handle this problem, we propose an Adaptive Staleness-aware Momentum Accelerated AFL (ASMAFL) algorithm to reduce the resources consumption of heterogeneous wireless communication EC (WCEC) scenarios, as well as decrease the negative impact of Non-IID data for model training. Specifically, we first introduce the staleness-aware parameter and a unified momentum gradient descent (GD) framework to reformulate AFL. Then, we establish global convergence properties of AFL, derive an upper bound on AFL convergence rate, and find that the bound is related to the staleness-aware parameter and Non-IIDness. Next, we formulate the bound into a minimization problem of resource consumption under given model accuracy, and the corresponding staleness-aware parameter of devices will be recomputed after each asynchronous aggregation to eliminate the differences of local models’ contribution to global model aggregation. Finally, extensive experiments are carried out to validate the superiority of ASMAFL in model accuracy, convergence rate, resources consumption, Non-IID issue, etc.

I. Introduction

The rapid development of the Internet of Things (IoT) era makes a large number of data be generated on the edge network [1]. Therefore, the traditional paradigm of uploading data to the cloud center server is no longer applicable. Edge computing (EC) [2] has emerged as a promising solution to address the need for pushing computation and storage resources closer to edge devices, enabling local processing of tasks. However, the implementation of privacy-related regulations (such as GDPR [3]) indicates that people’s privacy awareness has increased. Therefore, it poses an obstacle for edge servers to collect user data for model training.