How Reinforcement Learning is Helping to Solve Internet-of-Underwater-Things Problems

We present a review of how reinforcement learning (RL) is helping to tackle some of the most challenging problems in the Internet-of-underwater-things (IoUTs). Scientists estimate that 50-80 percent of atmospheric oxygen comes from the ocean, implying that life on earth depends heavily on clean and healthy oceans. This huge significance of the ocean in supporting life on earth is motivating the use of artificial intelligence (AI) and machine learning (ML) tools to create a sustainable marine ecosystem. We briefly review the RL paradigm, its categorisations and RL algorithms developed to solve important problems in IoUTs. New literature keeps emerging that show innovative applications of RL in underwater communications and networking that far outperform conventional solutions and other ML-based methods. Due to its online learning nature, RL is particularly useful for decision making in dynamic environments such as underwater where the communication channel is stochastic and rapidly varying. We explore the applications of RL in IoUTs, showing different classes of IoUTs problems and high-lighting RL algorithms that are tailored to solving them. Despite the significant progress that has made in the RL field, there are still many challenges and open research problems in the use of RL in IoUTs. We conclude the article with an outline of some of these challenges and suggest some ways forward.