Efficient Federated Learning for Cloud-Based AIoT Applications

As a promising method for central model training on decentralized device data without compromising user privacy, federated learning (FL) is becoming more and more popular in Internet-of-Things (IoT) design. However, due to limited computing and memory resources of devices that restrict the capabilities of hosted deep learning models, existing FL approaches for artificial intelligence IoT (AIoT) applications suffer from inaccurate prediction results. To address this problem, this article presents a collaborative Big.Little branch architecture to enable efficient FL for AIoT applications. Inspired by the architecture of BranchyNet which has multiple prediction branches, our approach deploys deep neural network (DNN) models across both cloud and AIoT devices. Our Big.Little branch model has two branches, where the big branch is deployed on cloud for strengthened prediction accuracy, and the little branches are used to fit for AIoT devices. When AIoT devices cannot make the prediction with high confidence using local little branches, they will resort to the big branch for further inference. To increase both prediction accuracy and early exit rate of Big.Little branch model, we propose a two-stage training and coinference scheme, which considers the local characteristics of AIoT scenarios. Comprehensive experiment results obtained from a real AIoT environment demonstrate the efficiency and effectiveness of our approach in terms of prediction accuracy and average inference time.

I. Introduction

Along with the prosperity of artificial intelligence (AI), deep learning (DL) techniques are increasingly deployed in Internet-of-Things (IoT) domains, such as commercial surveillance, autonomous driving, and robotics, where the requirements of model prediction accuracy and real-time response are of crucial importance [1]–[3]. However, due to limited computing and memory resources of AI IoT (AIoT) devices, both requirements cannot be guaranteed [4]–[6]. To extend the processing capabilities of AIoT devices for higher prediction accuracy, more and more AIoT applications rely on cloud computing by offloading some part of computation-intensive tasks to remote cloud servers. Therefore, the combination of strong cloud computing platforms and weak AIoT devices is becoming an emerging paradigm for large-scale AIoT design [7]–[9]. However, no matter whether the role of cloud servers in AIoT applications is for DL model training or inference, it is required that AIoT devices need to send their private data to cloud servers, where the concerns of user privacy and network latency cannot be neglected.