Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraints

Federated learning systems facilitate the training of global models across large numbers of distributed edge-devices with potentially heterogeneous data. Such systems operate in resource constrained settings with intermittent client availability and/or time-varying communication constraints. As a result, the global models trained by federated learning systems may be biased towards clients with higher availability. We propose Federated Averaging Aided by an Adaptive Sampling Technique (F3ast), an unbiased algorithm that dynamically learns an availability-dependent client selection strategy which asymptotically minimizes the impact of client-sampling variance on the global model's convergence, enhancing performance of federated learning. The proposed algorithm is tested in a variety of settings for intermittently available clients operating under communication constraints, and its efficacy demonstrated on synthetic data and realistically federated benchmarking experiments using CIFAR100 and Shakespeare datasets. We report up to 186% and 8% accuracy improvements over FedAvg, and 8% and 7% over FedAdamon CIFAR100 and Shakespeare, respectively.

I. Introduction

Federated learning (FL) has emerged as an attractive framework in edge learning to train models when the data is distributed among edge devices and must remain local due to resource constraints and/or privacy concerns. The edge-device networks in FL could comprise millions of clients [1] whose feedback might include model updates that are on the order of 100 Mb. For example, neural network for image recognition tasks VGG-16 [2] has 160 M parameters and weights resulting in updates of size 526Mb when using 32 b encoding.