A PATE-based Approach for Training Graph Neural Networks under Label Differential Privacy

As a standard solution to the problem of private deep learning, differential privacy (DP) is widely used in graph neural networks (GNNs) to protect sensitive information about the input graph data. However, most existing DP algorithms for GNNs protect the privacy of every attribute for each node. This results in the need for injecting a large amount of noise, making these methods significantly underperform their non-private counterparts. We argue that in some practical scenarios, node labels serve as the only or the most sensitive attribute, where label differential privacy, a more fine-grained notion of differential privacy that only protects the labels is more appropriate. To better capture these scenarios and improve the trade-off between data privacy and model accuracy, we propose a novel method of training GNNs under label differential privacy. Instead of naively adding noise to the node labels before training the GNN, our method follows the strategy of Private Aggregation of Teacher Ensembles (PATE) to generate differentially private node labels with both high accuracy and strong privacy guarantee. We also propose a label denoising module that takes advantage of the graph structure to further improve the accuracy of the trained model. Additionally, our method is model-agnostic, making it applicable to any GNN architecture. We evaluate its performance on two commonly used benchmark datasets and demonstrate its capability to learn high-performance models while ensuring privacy.

I. Introduction

The success of deep learning models in image and text domains has spurred a growing interest in extending these methods to graph-structured data, leading to the rise of Graph Neural Networks (GNNs) [1]. Due to their ability to directly leverage the graph structure information, GNNs have shown superior performance in various applications, including traffic prediction [2], social network analysis [3], and recommendation systems [4]. However, similar to deep learning models trained on images or text, GNNs may unintentionally reveal sensitive information about their training data [5]. This raises concerns about security and privacy when training with sensitive graph data.