FedSyL: Computation-Efficient Federated Synergy Learning on Heterogeneous IoT Devices

As a popular privacy-preserving model training technique, Federated Learning (FL) enables multiple end-devices to collaboratively train Deep Neural Network (DNN) models without exposing local privately-owned data. According to the FL paradigm, resource-constrained end-devices in IoT should perform model training which is computation-intensive, whereas the edge server occupied with powerful computation capability only performs model aggregation. Due to the above unbalanced computation pattern, IoT-oriented FL is time-consuming and inefficient. In order to alleviate the computation burden of end-devices, recent countermeasures introduce the edge server to assist end-devices in model training. However, existing works neither efficiently address the computation heterogeneity across end-devices nor reduce the leakage risk of data privacy. To this end, we propose a Federated Synergy Learning (FedSyL) paradigm which innovatively strikes a balance between training efficiency and data leakage risk. We explore the complicated relationship between the local training latency and multi-dimensional training configurations, and design a uniform training latency prediction method by applying the polynomial quadratic regression analysis. Additionally, we design the optimal model offloading strategy with the consideration of resource limitation and computation heterogeneity of end-devices, so as to accurately assign capability=matched device-side sub-models for heterogeneous end-devices. We implement FedSyL on a real test-bed comprising multiple heterogeneous end-devices. Experimental results demonstrate the superiority of FedSyL on training efficiency and privacy protection.

I. Introduction

In recent years, various Deep Neural Network (DNN)-driven applications have been widely applied in Internet of Things (IoT), such as activity recognition and real-time monitoring. Massive data is generated by end-devices in IoT and utilized to train DNN models [1]. Traditional solutions transmit privately-owned data of end-devices to the central server for analyzing and training. This central training manner inevitably poses threats to data privacy and high-communication overhead. Compared with the cloud server, the edge server is geographically close to end-devices and provides relatively reliable and short-distance transmission [2]. Although utilizing the edge server to collect data and train models can significantly reduce communication cost, the privacy concern remains unsolved.