PyHENet: A Generic Framework for Privacy-Preserving DL Inference Based on Fully Homomorphic Encryption

Deep learning inference provides inference service by service provider with model for client with input of personal data. Due to the huge commercial value inside, on one hand, both client's original data and inference output should be kept secret from others, even including service provider. On the other hand, service provider's model should be kept secret, especially from his competitor. Current research on privacy-preserving deep learning inference focuses on building models with specific data. This paper proposes a generic framework PyHENet of privacy-preserving deep learning inference based on Pytorch and lattice-based FHE, such that crypto library can be flexibly embedded into network. Firstly, raw data is encrypted by lattice-based FHE and uploaded to service provider. Secondly, convolutional computation over float-point ciphertext data is proposed for service provider to execute low accuracy loss inference with aided parallel method SIMD. Thirdly, inference result in ciphertext format is sent back to client for decryption. To improve efficiency, inference procedure can be further divided into two phases. All the computations during the second phase are in plaintext format with GPU acceleration, while keeping the first phase unchanged. Using the same model and parameters, the relative accuracy of PyHENet is almost 100% compared to the plaintext inference. This paper is the first to propose a general framework of neural networks for fully homomorphic cryptographic inference, and is based on mainstream deep learning frameworks, which is both secure and more conducive to development.

I. Introduction

With the development of the big data and network, privacy preserving issues have become increasingly prominent. On one hand, countries and organizations are strengthening the protection of data privacy. The EU 's General Data Protection Regulation (GDPR), which entered into force in 2018, has created a new era of privacy preserving. The California's CCPA may push privacy preserving one step further by its influence. In December 2019, Twitter announced the creation of a new “Privacy Center” to update its privacy policy. Mozilla announced that it plans to enable users can delete their own data from Mozilla's servers started from January 1, 2020. Private data has been protected by laws since then. As planned by Mozila, users' privacy data should be in their own hands. This poses new challenges for industries that process personal privacy data, such as medical clinical diagnostics.