Cooperative and Dynamic UAV-Based Mobile Sink Deployment using Deep Reinforcement Learning

In maritime emergency response operations, autonomous underwater vehicles (AUVs) can perform underwater search and transmit emergency data in real-time. To collect the sensed data from AUVs across the water-air interface, the traditional method is to deploy surface relays. However, the deployment of surface relays has some disadvantages, such as poor and imbalanced communication coverage of real-time connections with AUVs. Since the deployment of static relays may limit the dynamic search area of AUVs, whereas the deployment of free surface drifting relays may disrupt the connections while the relays are drifting away with the ocean current. To address these challenges, in this paper, we propose a novel method by leveraging the flexibility of multiple unmanned aerial vehicles (UAVs) equipped with underwater hydrophones and design a dynamic deployment scheme based on a multi-agent deep deterministic policy gradient (MADDPG) approach. By this scheme, multiple UAVs are able to deploy in a cooperative manner to deal with the coverage imbalance issue and provide maximum coverage service. The numerical simulations demonstrate that the proposed scheme is effective in terms of communication coverage, fairness, and energy consumption.

I. Introduction

The oceans occupy nearly 71% of the Earth’s surface and provide abundant resources for human survival [1], [2]. In ocean exploration, marine emergency events occur frequently, such as tsunamis, shipwrecks and oil spills. The occurrence of maritime accidents not only causes great losses to human beings but also destroys the marine ecosystem. Due to the complexity of the marine environment, it is common to establish surface base stations to realize underwater communications. However, when carrying out maritime emergency response missions, the sea surface base station is unable to achieve fair coverage, which in turn leads to high latency or failed transmission of underwater sensed data.