Study on Hyperparameter Adaptive Federated Learning

Federated Learning (FL) is a decentralized machine learning framework that distributes training data across mobile devices to learn a shared model by aggregating updates from local computations. FL protects the privacy of the user. In most federated learning algorithms, they usually experiment with different hyperparameter values to select a static hyperparameter value with better results. Therefore, one challenge associated with federated learning is the determination of the hyperparameter values in the algorithm, which limits the global model from providing good performance in each update. To address this issue, we propose a personalized FL algorithm that dynamically adjusts the hyperparameters (HAFed), using a log function to change the hyperparameter values in each round of updates, which helps the model to obtain different hyperparameter values in each update to address the limitation of choosing static hyperparameters. In theory, changing the value of the hyperparameters in each update by taking log for the parametric measure of the gap between client-side models allows the hyperparameters to be varied effectively over a range. In our experiments, we verified that HAFed outperforms other personalization algorithms.