Survey of Personalization Techniques for Federated Learning

Federated learning enables machine learning models to learn from private decentralized data without compromising privacy. The standard formulation of federated learning produces one shared model for all clients. Statistical heterogeneity due to non-IID distribution of data across devices often leads to scenarios where, for some clients, the local models trained solely on their private data perform better than the global shared model thus taking away their incentive to participate in the process. Several techniques have been proposed to personalize global models to work better for individual clients. This paper highlights the need for personalization and surveys recent research on this topic.

I. Introduction

Many datasets are inherently decentralized in nature and are distributed across multiple devices owned by different users. Traditional machine learning settings involve aggregating data samples from these users into a central repository and training a machine learning model on it. This movement of data from local devices to a central repository poses two key challenges. Firstly, it compromises the privacy and security of the data. Policies such as the General Data Protection Regulation (GDPR) [1] and Health Insurance Portability and Accountability Act (HIPAA) [2] stipulate provisions that make such movement difficult. Secondly, it imposes communication overheads which, depending on the setting, may be prohibitively expensive.