FedMPT: Federated Learning for Multiple Personalized Tasks Over Mobile Computing

Federated learning (FL) is a privacy-preserving collaborative learning framework that can be used in mobile computing where multiple user devices jointly train a deep learning model without uploading their data to a centralized server. An essential issue of FL is to reduce the significant communication overhead during training. Existing FL schemes mostly address this issue regarding single task learning. However, each user generally has multiple related tasks on the mobile device such as multi-content recommendation, and traditional FL schemes need to train an individual model per task which consumes a substantial number of resources. In this work, we formulate an FL problem with multiple personalized tasks, which aims to minimize the communication cost in learning different personalized tasks on each device. To solve the formulated problem, we incorporate multi-task learning into FL which trains a model for multiple tasks concurrently and propose an FL framework named FedMPT. FedMPT modifies the efficient acceleration algorithm and quantization compression strategy delicately to achieve superior performance regarding the communication efficiency. We implement and evaluate FedMPT on two datasets, Multi-MNIST and CelebA, in the FL environment. Experimental results show that FedMPT significantly outperforms the traditional FL scheme considering communication cost and average accuracy.

I. Introduction

With the advancement of the Internet of Things (IoT), intelligent mobile devices equipped with various sensors can collect and process data at unprecedented scales [1], [2], [3]. The generated massive volumes of data constitute a great source for training deep learning models in mobile computing systems [4], such as enhancing driving safety [5] and inferring emotional states [6]. However, users with mobile devices are usually not willing to upload their potentially privacy-sensitive data directly, which may cause severe private information leakages, such as personal position and gender. Furthermore, though deep learning has shown state-of-the-art performance in various application scenarios such as Computer Vision (CV) [7], [8], [9] and Natural Language Processing (NLP) [10], it is computationally expensive generally and requires enormous amounts of training data, especially with the trend of increasing model size, which is impractical for mobile devices.