DegaFL: Decentralized Gradient Aggregation for Cross-Silo Federated Learning

Federated learning (FL) is an emerging promising paradigm of privacy-preserving machine learning (ML). An important type of FL is cross-silo FL, which enables a moderate number of organizations to cooperatively train a shared model by keeping confidential data locally and aggregating gradients on a central parameter server. However, the central server may be vulnerable to malicious attacks or software failures in practice. To address this issue, in this paper, we propose $\mathtt{DegaFL} $, a novel decentralized gradient aggregation approach for cross-silo FL. $\mathtt{DegaFL} $ eliminates the central server by aggregating gradients on each participant, and maintains and synchronizes gradients of only the current training round. Besides, we propose $\mathtt{AdaAgg} $ to adaptively aggregate correct gradients from honest nodes and use HotStuff to ensure the consistency of the training round number and gradients among all nodes. Experimental results show that $\mathtt{DegaFL} $ defends against common threat models with minimal accuracy loss, and achieves up to $50\times$ reduction in storage overhead and up to $13\times$ reduction in network overhead, compared to state-of-the-art decentralized FL approaches.

I. Introduction

In many real-world scenarios, such as e-commerce, clinical services, social networks, and the Internet of Things (IoT), data are distributed among devices or organizations and it is a common practice to gather these data and train a global model for intelligent services, such as recommendation and anomaly detection. Undoubtedly, it raises concerns about data ownership, privacy, security, and monopolies. Federated Learning (FL) [1] mitigates some of these concerns by training a global model without gathering confidential data from each participating node. Cross-silo FL [2], [3] is an important type of FL where a moderate number of organizations collectively train a model on a central parameter server provided by a third party. All participants assume the central server to be trusted and reliable. However, this assumption may not hold in practice. For example, the central server could be malicious, leading to poisoning the model [4], [5], [6], [7], or skewing the model by favoring particular participants [8], [9]. Besides, fatal crashes in the central server could lead to an accuracy drop, convergence time increase, or even training procedure abortion.