DAFL: Device-to-Device Transmissions for Delay-Efficient Federated Learning Over Mobile Devices

Federated learning (FL) over mobile devices is an emerging distributed learning paradigm for numerous delay sensitive applications. In FL, the training delay is composed of the computing and communication delay. Some of the participating mobile devices may have slow local computing or wireless communications, which results in high FL training delay. Intuitively, if fast devices help slow ones, the FL training delay can potentially be reduced. However, helping each other among devices requires frequent transmissions and may cause additional delay. Fortunately, we observe that device-to-device (D2D) transmission, a fast and direct transmission, may be applied between device pairs to mitigate the additional delay from frequent transmissions. Inspired by those observations, we develop the D2D transmission assisted FL (DAFL), a novel FL scheme to improve the training delay over mobile devices. Briefly, we first put the eligible mobile devices into pairs, assigning each pair to one of the four types of relation: 1) similar computing, large communication gap; 2) similar communication, large computing gap; 3) one with faster computing and the other with faster communication; and 4) one with both faster computing and communication. We design the process for each type of device pair to: 1) improve the transmission delay of each pair, by letting the fast device help with the model parameters transmission to the server and 2) improve the computing delay by splitting learning task between paired devices. The emulation results demonstrate that DAFL surpasses existing peer designs in terms of reducing training delay by more than 20%.

I. Introduction

Nowadays, federated learning (FL) is considered a promising distributed deep learning (DL) solution, which can yield efficient and privacy-preserving collaborative training over many devices [1]. Meanwhile, with ever-increasing computing capability, mobile devices can conduct the on-device training of more and more deep neural network (DNN) models. FL over mobile devices has prompted numerous applications, such as keyboard predictions [2], physical hazard detection in smart homes [3], health event detection [4], etc. A key obstacle to unleash the full potential of FL over mobile devices is the long training delay, especially for delay-sensitive mobile applications. One of the possible reasons behind this is the device heterogeneity among FL clients, which can be defined as diverse computing capabilities and communication conditions [5]. Among those mobile devices, “slow” performing ones are considered stragglers for the FL training. In FL, the server and other devices may need to wait for all the stragglers to finish their local updates. Therefore, the FL training delay is bottlenecked by such slow devices.