Mean-Field Multi-Agent Reinforcement Learning for UAV Assisted Secure Data Dissemination

Unmanned aerial vehicles (UAVs) can be deployed as aerial relays touring to disseminate data to ground users (GUs). The broadcasting nature of wireless propagation results in communication that may expose to potential malicious eaves-droppers (Eves). By leveraging the cooperation among UAVs, they can spatially detour the Eves to conquer the possible wire-tapping. However, the interaction among multiple UAVs leads to a huge communication overhead for sharing their states, which makes large-scale practical applications difficult. To simplify complex interactions, the paper proposes a decentralized multi-agent deep reinforcement learning (MADRL) with mean-field augment, where multi-UAV can be clustered into groups and novelty conduct virtual UAV agents to represent the mean-field state and actions, thus the interactions among UAVs can be significantly reduced and UAVs can learn distributively from their historic experiences. Therefore, multi-UAV's dissemination secure rate, as well as trajectory, and energy consumption can be decentralized and optimized with subjective to GUs' dissemination data size. Numeral simulations demonstrate the proposed mean-field augmented MADRL method can approach the same performance as other counterpart DRL approaches but with the advantage of the great reduction of communication overhead.

I. Introduction

Unmanned aerial vehicles (UAVs) assisted communication, with a high probability of line-of-sight (LoS) over air-to-ground (A2G) wireless links, has received great attention in both military and civilian applications [1]. However, the broadcasting quality of wireless communication makes it highly possible that it will be wiretapped by malicious eavesdroppers (Eves). Recently, physical layer security technology has become a promising solution for realizing confidentiality in wireless communication besides the traditional encryption technique. However, when multiple ground mobile users (GUs) and UAVs are involved in the system, the service assignment among UAVs and legalistic GUs, the UAVs' movement with consideration of secure rate maximization and propulsion energy-saving as well as the quality of service (QoS) of GUs, make the UAVs' trajectory optimization a challenge.