Optimize Scheduling of Federated Learning on Battery-powered Mobile Devices

Federated learning learns a collaborative model by aggregating locally-computed updates from mobile devices for privacy preservation. While current research typically prioritizing the minimization of communication overhead, we demonstrate from an empirical study, that computation heterogeneity is a more pronounced bottleneck on battery-powered mobile devices. Moreover, if class is unbalanced among the mobile devices, inappropriate selection of participants may adversely cause gradient divergence and accuracy loss. In this paper, we utilize data as a tunable knob to schedule training and achieve near-optimal solutions of computation time and accuracy loss. Based on the offline profiling, we formulate optimization problems and propose polynomial-time algorithms when data is class-balanced or unbalanced. We evaluate the optimization framework extensively on a mobile testbed with two datasets. Compared with common benchmarks of federated learning, our algorithms achieve 210× speedups with negligible accuracy loss. They also mitigate the impact from mobile stragglers and improve parallelism for federated learning.

I. Introduction

The past few years have witnessed an increasing migration of data-driven applications from the centralized cloud to mobile devices due to the rising privacy concerns. Originated from distributed learning [1], Federated Learning (FL) learns a centralized model where the training data is held privately by end users [2]–[10]. They compute local models in parallel and aggregate their updates towards a centralized parameter server. The server takes the average from the users, pushes the averaged model back to all the users as the initial point for the next iteration.