A Reinforcement-Learning-Based Beam Adaptation for Underwater Optical Wireless Communications

Underwater optical wireless communications (UOWCs) have recently appeared as an attractive solution for many applications, such as remote controlling and sensing due to its advantages, such as high transmission rate, ultrawide bandwidth, and low latency. However, due to the harsh underwater conditions, UOWC faces challenges, such as water absorption, scattering, and pointing-acquisition-and-tracking (PAT) problems. This is mainly due to the dynamicity existing underwater. Consequently, this leads to packet loss and hence deteriorates the reliability and link quality of such networks. Such a problem can affect the degree of connectivity and end-to-end (E2E) performance of the communication system. The existing solutions in the literature are based on predefined models, assuming full knowledge of the environment. However, such models do not optimally treat the dynamicity existing underwater. This article proposes novel beam adaptation methods based on reinforcement learning (RL) for point-to-point UOWC. The first method aims to optimize the light beamwidth; the second method focuses on adapting the beam orientation, whereas the last one optimizes both the light’s beamwidth and beam orientation. Our proposed RL-based solutions yield optimal positioning and beamwidth of the light source and improve the considered communication link’s success rate. They also guarantee better link quality in terms of signal-to-noise ratio (SNR) compared to the uncertainty disk static method for four different underwater environments, including pure seawater, clean ocean, coastal ocean, and turbid harbor.

I. Introduction

Nowadays, humans’ interaction with the underwater world is increasing, which makes underwater communication an attractive field to exploit. This type of communication became crucial in many areas, such as the remote control of instruments in oceans, pollution monitoring, oceanography research, oil-controlling, maintenance, etc. These applications explain the remarkable increase of devices deployed underwater lately. Such devices require high bandwidth and capacity while keeping a low power consumption. Previously, acoustic and radio-frequency (RF) waves were considered for underwater communications as they offer cheaper and more flexible communications. Many papers in the literature have focused on underwater acoustic communications [1]–[3]. Moreover, RF underwater communication has been a widely investigated topic as well [4]–[6]. However, acoustic waves suffer from high latency and limited bandwidth [7], [8], while RF waves suffer from high power consumption and severe attenuation [9], [10]. Given the limitations of underwater acoustic and RF communications, we recently witnessed the upsurge of a new means of communication, namely, underwater optical wireless communication (UOWC) [11], [12]. The aforementioned technology is based on exchanging information in submarine environments employing light waves. Such communication offers many advantages, such as higher data rates than acoustic waves, lower power consumption than RF waves, a simpler implementation, and lower computational complexity. UOWC has been widely used in scientific, environmental, civil, and military applications. Such applications include marine exploration, detection of human-made and natural environmental disasters, and submarine communications. In order to guarantee reliable and robust underwater communications, basic knowledge of the environment is needed. Nevertheless, accurately modeling underwater optical wireless channels is challenging due to the harsh underwater conditions. Unlike terrestrial optical wireless communications, underwater light propagation undergoes many losses mainly caused by absorption and scattering. This shortens the coverage range of the optical base station (OBS), defined by the underwater area where the nodes can communicate with the OBS. Moreover, it degrades the link quality of the UOWC and the connectivity of the network can be affected. The transmitted light can also be blocked by the passage of marine creatures and plants on its way. Underwater OBSs are not fixed and can change their position or orientation due to turbulence or convulsive water movement. This can lead to misalignment problems and consequently lead to packet loss. The retransmission of packets deteriorates these networks’ performance by introducing problems, such as the extension of transmission delays, the increase of power consumption, and congestion problems. These aforementioned problems make reliability, which is the probability of performing a successful communication, a fundamental challenge for such communications.