Convergence Analysis and System Design for Federated Learning Over Wireless Networks

Federated learning (FL) has recently emerged as an important and promising learning scheme in IoT, enabling devices to jointly learn a model without sharing their raw data sets. As FL does not collect and store the data centrally, it requires frequent model exchange through the wireless network. However, since the aggregation in FL can be partially participated with synchronized frequency, its communication pattern is different from the conventional network. Therein, limited bandwidth and package loss restrict interactions in training. Thus, the network scheduling could largely affect the FL convergence. To figure out the specific effects, we analyze the convergence rate of FL regarding the joint impact of communication and training. Combining it with the network model, we formulate the optimal scheduling problem for FL implementation. The theoretical results could guide the hyper-parameter design in the network and explain the principle of how the wireless communication could influence the FL training process.

I. Introduction

With the emergence of distributed data sets and the rapid growth of computing capability, distributed learning has become a promising mode for deployment of large-scale machine learning [1]. Under such circumstances, due to data privacy and limited communication resources, it is very difficult to transmit all the raw data sets to a central server for learning, especially for applications with widely distributed clients. Thus, to implement machine learning with high efficiency, researchers started to focus on the distributed learning schemes. In [2], the authors proposed the federated learning (FL) scheme for distributed data sets. Such a technique allows applications to collectively reap the benefits of shared models trained from the rich data while avoiding the need of central data collection.