Partially Encrypted Multi-Party Computation for Federated Learning

Multi-party computation (MPC) allows distributed machine learning to be performed in a privacy-preserving manner so that end-hosts are unaware of the true models on the clients. However, the standard MPC algorithm also triggers additional communication and computation costs, due to those expensive cryptography operations and protocols. In this paper, instead of applying heavy MPC over the entire local models for secure model aggregation, we propose to encrypt critical part of model (gradients) parameters to reduce communication cost, while maintaining MPC’s advantages on privacy-preserving without sacrificing accuracy of the learnt joint model. Theoretical analysis and experimental results are provided to verify that our proposed method could prevent deep leakage from gradients attacks from reconstructing original data of individual participants. Experiments using deep learning models over the MNIST and CIFAR-10 datasets empirically demonstrate that our proposed partially encrypted MPC method can reduce the communication and computation cost significantly when compared with conventional MPC, and it achieves as high accuracy as traditional distributed learning which aggregates local models using plain text.

1. Introduction

Machine learning, especially deep learning, has made significant breakthroughs in many domains of science, business and government, such as manufacturing, transportation, finance, and healthcare [1], [2]. The centralised learning mainly contributes to these remarkable successes on large-scale datasets. With the popularity of modern technologies of edge computing [3] and the Internet of Things [4], [5], machine learning has witnessed a dramatic change in the way it computes. Data in many real-world scenarios are naturally distributed and owned by different organisations/users. Due to the competition of different organisations, data privacy security, and administrative regulations, it is almost impossible to upload the data across countries and institutions for centralised learning [2], [6].