Fusion of Current and Historical Knowledge for Personalized Federated Learning

Data heterogeneity poses a significant challenge in the realm of federated learning. Personalized federated learning has emerged as a crucial solution to mitigate this challenge. In these methods, the local model is initially replaced with a global model, followed by personalized processing to adapt to the local data. However, a substantial disparity in knowledge representation between the global and local models can lead to the loss of previously acquired knowledge by the local model, which is commonly referred to as catastrophic forgetting. To tackle this issue, we propose a method named FedKML, which utilizes knowledge mutual learning to achieve the fusion of current and historical knowledge. Our method involves preserving the trained local model as a historical local model for each client, thereby retaining valuable personalized knowledge from the past. By facilitating mutual learning between the current and historical local models, the local model can effectively integrate both the current generalized knowledge and the historical personalized knowledge. Extensive experiments showcase the superiority of our method compared to state-of-the-art methods.