Deep Reinforcement Learning Based Energy Efficient Underwater Acoustic Communications

Due to the unique channel characteristics and the difficulty of battery charging, energy efficient transmission is a critical topic in underwater acoustic communications (UACs). This paper considers the transmit frequency and power selection of a single link in the UAC, aiming to achieve best tradeoffs between packet delivery ratio and energy consumption, i.e., maximize the energy efficiency. Unlike traditional optimization approaches which rely on system statistics, we employ the deep reinforcement learning (DRL) technique to learn the optimal transmission strategy on the fly, without requiring any prior environmental information. Since two different actions need to be selected, traditional DRL algorithms are no longer applicable to our problem. Motivated by this, we put forth a new DRL algorithm that decides and evaluates two types of actions separately, referred to as two-action selection-based deep Q-network (TAS-DQN). The numerical results demonstrate that TAS-DQN outperforms Q-learning and original DQN, and achieves near optimal energy efficiency of the network.