Towards Secrecy Energy-Efficient RIS Aided UAV Network: A Lyapunov-Guided Reinforcement Learning Approach

Unmanned aerial vehicles (UAVs) are integrated into existing networks to enhance coverage, increase network capacity and provide ubiquitous access service. However, the channel in the UAV network is prone to noise and interference due to the complex environments. Reconfigurable intelligent surface (RIS), as an emerging technology in recent years, can be applied to the UAV network to establish the transmission environment by intelligibly adjusting signal characteristics, which can achieve significant gains in coverage and spectral efficiency. Thus, we consider RIS aided UAV networks for virtual reality (VR) content transmission under the presence of eavesdroppers, and maximize the time average sum secrecy energy efficiency (SEE) via adjusting UAV trajectory, beamforming matrix of UAV and RIS jointly by the deep reinforcement learning (DRL) approach. To eliminate the time correlation and the coupling of variables, we propose a Lyapunov guided decay twin-delayed deep deterministic policy gradient (TD3) scheme to tackle the decoupled problem. Simulations demonstrate the effectiveness of the proposed scheme and its outperformance in SEE compared with other benchmarks.

I. Introduction

With the development of next generation wireless networks, unmanned aerial vehicles (UAVs) are widely deployed to support various human requirements [1]. UAVs can provide line-of-sight (LoS) links when working as base stations or relay nodes. However, a large scale of interferences caused by dense buildings and complex environments make it difficult to accurately characterize channels, which place a severe impact on users' downlink rates.