Energy Optimization for Federated Learning on Consumer Mobile Devices With Asynchronous SGD and Application Co-Execution

Federated learning relies on distributed training on mobile device. The previous research mainly focuses on addressing the heterogeneity from computation and data distributions. As battery life remains to be the performance bottleneck on mobile devices, energy consumption from the persistent training tasks poses great challenges. In this paper, we propose an online scheduler to optimize energy usage by leveraging application co-execution and asynchronous gradient updates. Motivated by a series of empirical studies, we find that placing the training process in the background while co-running a foreground application gives the system a large energy discount. Based on these findings, we first study an offline baseline assuming all the application occurrences are known in advance, and propose a dynamic programming solution. Then we propose an online scheduler using the Lyapunov framework to exploit the energy-staleness/slowdown trade-offs and prove the convergence at the rate of $1/\sqrt{K}$. We conduct extensive experiments on a mobile testbed with devices from different vendors. The results indicate 10-30% energy saving and much faster convergence compared to FedAvg and FedProx with 3-4% higher testing accuracy under the non-IID data setting. The design is also validated in terms of resource utilization, memory bandwidth and Frame-Per-Second rates.

I. Introduction

These years, we are witnessing our planet warming up at an unprecedented rate, causing irreversible change to the climate [1]. Unfortunately, the recent advance in AI plays an increasing role in this tragedy [2]. For example, training a large transformer model would generate the same amount of carbon emissions as five fuel vehicles in their entire lifetime [3]. The study in [4] shows that 40% of the energy comes from centralized cooling while training is performed in data centers. On the other hand, Federated Learning (FL) rises as a new paradigm to preserve user privacy by conducting training on distributed mobile/edge devices [6], [7], [9], [10]. It is shown that federated learning potentially offers better energy efficiency because centralized cooling is no longer required [5].