MURAO: A multi-level routing protocol for acoustic-optical hybrid underwater wireless sensor networks

In the past decade, underwater acoustic sensor networks (UW-ASNs) have been studied broadly in various aquatic applications, enabling humans to observe and explore the vast underwater domain. Although acoustic underwater communications are able to support long-range and low-bandwidth applications, the capabilities of UW-ASN are greatly limited by the long delay and low data rate of acoustic communications. Underwater free-space optical communication is a potential alternative solution. However, it has short transmission ranges and requires dense deployment. In this paper, we propose a novel acoustic-optical hybrid architecture for underwater wireless sensor networks, and a multi-level Q-learning based routing protocol, MURAO, for such networks. The network is physically partitioned into several groups and logically divided into two layers. The upper-layer group leaders supervise the routing in the lower layer, and the lower-layer group members carry out the actual data packet routing. Because upper-layer group leaders have a boarder view of the network and all the groups are able to carry out the learning process concurrently, the performance of routing is greatly improved compared to the flat Q-learning-based routing. The experiment results show that MURAO is more robust to changes of network topology, and achieves much higher delivery rates as well as shorter delays in a dynamic network than the flat Q-learning routing.