A Federated Learning Aggregation that Integrates Clustering and Momentum Correction Based on Historical High-Quality Gradients

Nowadays, federated learning emerges as a new technique to preserve data privacy in distributed machine learning by limiting local data to the client and only aggregating model parameters from multiple parties on the server side. However, it also faces consequent threats from malicious clients, and some methods have been proposed without sufficient defense capability and efficient aggregation. To address these problems, we propose a novel aggregation algorithm, which incorporates PCA operation and clustering to select the excellent client in each round; and brings in a momentum-accelerated historical quality gradient-based approach to increase the weight when the performances of the global model are consistently degraded, which can achieve faster convergence on the basis of ensuring that the update direction of the global model is not manipulated by malicious nodes. Experimental results on the MNIST datasets show that our algorithm is only 0.93% lower than FedAvg under a no-attack scenario, and exhibits faster convergence compared to Krum, Multi-Krum, and Trimmed Mean Aggregation, and the accuracy improves 10.68%,35.83%,19.65% under Black-Box Edge Attack, Sign Flipping Attack, and Same Value Attack.