An Adaptive MAC Protocol for Underwater Acoustic Networks Based on Deep Reinforcement Learning

Underwater acoustic networks (UWANs) faces many challenges, such as long propagation delay, complex dynamic environment and limited node energy. To address these challenges, this paper uses long propagation delay characteristics of UWANs to develop a media access control (MAC) protocol based on Dueling Double Deep Q-network (D3QN) algorithm with delay reward, called DR-D3QN-MA protocol, which aims to allocate time slots reasonably for intelligence node. The DR-D3QN-MA protocol uses the D3QN algorithm to enable the network to quickly converge to the optimal value function and improve the calculation accuracy of the Q value. At the same time, a training mechanism based on variance characteristics is proposed to make intelligence node better adapt to underwater dynamic environment and reduce node energy consumption. The simulation results show that DR-D3QN-MA protocol can coexist with other protocols, and the intelligence node can use the idle time slot. Compared with the same type of Delay-Reward Deep Reinforcement Learning Multiple Access (DR-DLMA) protocol, DR-D3QN-MA protocol has better performance in system throughput, dynamic adaptability and energy consumption.