A Federated Learning Aggregation that Integrates Clustering and Momentum Correction Based on Historical High-Quality Gradients

Nowadays, federated learning emerges as a new technique to preserve data privacy in distributed machine learning by limiting local data to the client and only aggregating model parameters from multiple parties on the server side. However, it also faces consequent threats from malicious clients, and some methods have been proposed without sufficient defense capability and efficient aggregation. To address these problems, we propose a novel aggregation algorithm, which incorporates PCA operation and clustering to select the excellent client in each round; and brings in a momentum-accelerated historical quality gradient-based approach to increase the weight when the performances of the global model are consistently degraded, which can achieve faster convergence on the basis of ensuring that the update direction of the global model is not manipulated by malicious nodes. Experimental results on the MNIST datasets show that our algorithm is only 0.93% lower than FedAvg under a no-attack scenario, and exhibits faster convergence compared to Krum, Multi-Krum, and Trimmed Mean Aggregation, and the accuracy improves 10.68%,35.83%,19.65% under Black-Box Edge Attack, Sign Flipping Attack, and Same Value Attack.

I. Introduction

Amid the ongoing evolution of traditional centralized machine learning, due to the need for large-scale collection of raw data from end-users, end-users are at risk of losing control over their own data. Moreover, they face serious privacy leakage risks during data transmission and storage, thereby gradually exposing issues such as privacy protection and data silos^[1]. To address this situation, federated learning^[2–3], a distributed machine learning technique, has been introduced. Its advantage lies in leveraging the characteristics of distributed data, enabling data to remain local. It achieves model updates and iterations by aggregating parameters from multiple parties, collectively modeling while satisfying the privacy requirements of participating parties, thus addressing privacy protection and data sharing challenges.