Privacy-Preserving Deep Reinforcement Learning based on Differential Privacy

Deep reinforcement learning, with its extensive applications and remarkable performance, is emerging as a pivotal technology garnering researchers’ attention. During the training process, there are frequent interaction and data exchange between agents and the environment, and the interaction information during training is closely tied to the training environment. Consequently, this process introduces a high risk of environmental privacy leakage. Malicious third parties may potentially steal state transition matrix or environmental information about the application domain of agent training, resulting in the compromise of user privacy. To address this issue, we propose novel differentially private value-based and policy-based deep reinforcement learning algorithms. Our methods have an advantage of being adaptable to various environmental privacy concerns. We also evaluate them in a customized experimental environment. Comparative experiments are conducted between the original and differentially private versions of the algorithms. The results indicate that our proposed approach can provide differential privacy protection to environmental information with minimal impact on algorithm performance, ultimately achieving a good balance between privacy and utility.