Non-IID Federated Learning Based on Global Knowledge Sharing Without Out-of-Domain Data

Federated learning (FL) supports collaborative training of deep learning models on distributed data. The non-independent and identically distribution (non-IID) of user data brings great challenges to FL, which may cause the global model to converge slowly or even deviate from the optimal solution. As an effective method to solve the data non-IID problem, knowledge distillation often relies on public proxy datasets for model training, which violates the original intention of FL to protect privacy. In this paper, we propose a global knowledge sharing algorithm without out-of-domain data to solve FL with non-IID data. Firstly, to ensure that the data remains within the domain and to constrain the deviation of update directions between the local and global models, we propose a novel teacher-student network structure, which utilizes a personalized global knowledge transfer method to supervise local training. Secondly, we also design different knowledge transfer loss functions to maximize the use of global knowledge and prevent overfitting of local data. Finally, in order to continuously correct the update direction of the local model and improve the accuracy of the global model, the shared knowledge model on the client is updated using the exponential moving average (EMA) method. In order to evaluate, we manually constructed the non-IID dataset in the real scene. Experiments show that compared with the existing methods, the global model accuracy of the SKH-FL algorithm proposed in this paper is improved by 5.07%.