RDP-GAN: A Rényi-Differential Privacy Based Generative Adversarial Network

Generative adversarial networks (GANs) have attracted increasing attention recently owing to their impressive abilities to generate realistic samples with high privacy protection. Without directly interacting with training examples, the generative model can be used to estimate the underlying distribution of an original dataset while the discriminator can examine model quality of the generated samples by comparing the label values with training examples. In considering privacy issues in GANS, existing works focus on perturbing the parameters and analyzing the corresponding privacy protection capability, and the parameters are not directly exchanged between the generator and discriminator in GANs. Thus, in this work, we propose a Rényi-differentially private-GAN (RDP-GAN), which achieves differential privacy (DP) in a GAN by carefully adding random Gaussian noise to the value of the exchanged loss function during training. Moreover, we derive analytical results characterizing the total privacy loss under the subsampling method and cumulative iterations, which show its effectiveness for the privacy budget allocation. In addition, in order to mitigate the negative impact of injecting noises, we enhance the proposed algorithm by adding an adaptive noise tuning step, which will change the amount of added noise according to the testing accuracy. Through extensive experimental results, we verify that the proposed algorithm can achieve a better privacy level while producing high-quality samples compared with a benchmark DP-GAN scheme based on noise perturbation on training gradients.

1 Introduction

Recent technological advancements are transforming the ways in which data are created and processed. With the advent of the artificial intelligence (AI), data information is utilized with ground breaking successes in areas such as computer vision, natural language processing, voice recognition, etc. The great success of AI is owing to the development of powerful computing processor and the availability of massive data for training the neural networks. However, there exist domains where the accessibility of this huge data is not fully granted. For example, the sensitive medical data are usually not open-access in most countries. Thus, building a high-quality analytical model remains to be challenging at present. At the same time, data privacy has become a growing concern for clients. In particular, the emergence of centralized searchable data repositories has made the leakage of private information an urgent social problem, e.g., health conditions, travel information, and financial data. Furthermore, the diverse set of open data applications, such as census data dissemination and social networks, place more emphasis on privacy concerns. In such practices, the access to real-life datasets may cause information leakage even in pure research activities. Consequently, privacy preservation has become a critical issue.