Robust Model-based Reinforcement Learning USV System Guided by Lyapunov Neural Networks

This paper explores the potential of Lyapunov function approximated by neural networks in unmanned surface vehicles (USV) control problem. A novel model-based reinforcement learning method, Lyapunov filtered probabilistic model predictive control (LFPMPC) is proposed to explore the USV control policy under the guidance of Lyapunov neural networks. The USV system based on LFPMPC is developed and evaluated by a USV simulator driven by real boat data in position-keeping task with various environmental disturbances. Taking the output of Lyapunov neural networks as one metric of the system robustness in the cost function, the proposed approach demonstrated significant superiorities in not only control stability against disturbances but also learning capabilities of the system model compared with the baseline approach without Lyapunov neural networks.

I. Introduction

Unmanned surface vehicles (USV) have been quickly developed in recent years to tackle both the shortage of skilled professionals and the efficiency of operation in marine shipping industry [1]. A wide range of technologies from traditional control methods to learning-based approaches were proposed to control USV in various scenarios including navigation, collision avoidance and position control [2]–[6]. On the other hand, although safety is a core issue in robot control [7], it is less addressed in USV system where the description of safety and robustness strongly depends on not only human's prior knowledge of the target USV but also the manually selected features.