Robust Federated Learning: Maximum Correntropy Aggregation Against Byzantine Attacks

As an emerging decentralized machine learning technique, federated learning organizes collaborative training and preserves the privacy and security of participants. However, untrustworthy devices, typically Byzantine attackers, pose a significant challenge to federated learning since they can upload malicious parameters to corrupt the global model. To defend against such attacks, we propose a novel robust aggregation method—maximum correntropy aggregation (MCA), which applies the maximum correntropy criterion (MCC) to derive a central value from parameters. Different from the previous use of MCC for denoising, we utilize it as a similarity metric to measure parameter distribution and aggregate a robust center. Correntropy in MCC, with all even-order moments of the parameter, contains high-order statistical properties, which allows for a comprehensive capture of parameter characteristics, thus helping to prevent interference from attackers. Meanwhile, correntropy extracts information from the parameters themselves, without requiring the proportion of malicious attackers. Through the fixed-point iteration, we solve the optimization objective, demonstrating the linear convergence of the iteration formula. Theoretical analysis reveals the robustness aggregation property of MCA and the error bound between MCA and the global optimal solution, with linear convergence to the optimal solution neighborhood. By performing independent identically distribution (IID) and non-IID experiments on three different datasets, we show that MCA exhibits significant robustness under mainstream attacks, whereas other methods cannot withstand all of them.

I. Introduction

Fderated learning is a decentralized machine learning technique that aims to solve the problems of centralized data storage and privacy leakage in traditional centralized machine learning [1]. Throughout the federated learning process, devices or nodes perform model training locally and send their updated model parameters to a central server. The central server performs parameter aggregation on the submitted models. Subsequently, it distributes the aggregated model to each device as the initial model for the next round of training. Such an iterative procedure continues until training convergence is achieved. In this process, the interaction between the server and devices does not require sharing sensitive data, thus facilitating data privacy protection. Furthermore, the method of model aggregation promotes collaborative training between devices, thereby eliminating data silos [2]. Presently, federated learning has achieved extensive applications, including mobile applications deployed on millions of devices [3], [4] and brain image analysis [5].