Fed-MS: Fault Tolerant Federated Edge Learning with Multiple Byzantine Servers

Due to its decentralized framework and outdoor environments, federated edge learning (FEEL) faces significant vulnerability to malicious attacks within edge networks. Prevailing FEEL approaches typically hinge on a dependable parameter server (PS) to contend with the adversarial updates from Byzantine clients. Recognizing the inherent unreliability of PSs in edge networks, this paper delves into the security challenges of FEEL, specifically addressing Byzantine PSs. We present a Byzantine fault-tolerant FEEL algorithm, named Fed-MS, in which a multi-server technique along with a newly designed trimmed-mean-based model filter is employed. This combination ensures that each client can obtain a feasible global model for its local training, closely approximating a true model aggregated by benign PSs. Furthermore, we propose a sparse uploading strategy in Fed-MS to enhance communication efficiency for model aggregation to multiple PSs. Theoretical analysis demonstrates that, when Byzantine PSs are a minority, Fed-MS achieves an expected convergence speed of $O(1/T)$ with $T$ defined as the number of training rounds, akin to state-of-the-art works under non-Byzantine settings. Extensive experiments are conducted on the CIFAR-10 dataset with MobileNet V2 as the training model. The numerical results show that our Fed-MS can improve the model accuracy from 10% to at least 76% under the malicious attacks from Byzantine PSs. Our code is released at https://github.com/haoma2772/Fed-MS.

I. Introduction

As an interdisciplinary of artificial intelligence and networking, the federated edge learning (FEEL) enables large-scale and privacy-preserving machine learning (ML) on the edge scenarios and has witnessed significant growth in Industrial Internet of Things networks [1], [2]. Under the federated learning (FL) framework, multiple clients can obtain a high-quality global ML model by only sharing their local ML models instead of data to others, which protects the privacy of users. Besides, the edge-based parameter server (PS) provides the clients with fast and green model exchange. Thus, the FEEL has a higher efficiency and lower communication overhead, compared with the traditional cloud-based FL framework [3].