Approximate Optimal Indirect Control of an Unknown Agent Within a Dynamic Environment Using a Lyapunov-Based Deep Neural Network

An optimal control policy is derived for an indirect herding control problem in an environment with moving obstacles. A herding agent is tasked with influencing a target agent toward a goal location while evading obstacles. Because the target agent can not be directly controlled, and the unknown influence dynamics between the herder and target agent are coupled, a backstepping approach is used to enable shepherding behavior. The unknown dynamics are learned during task execution using a Lyapunov-based deep neural network. By using approximate dynamic programming, incorporating penalties for obstacle regions into the cost function and control policy, and using a state-following kernel for value function approximations the herding agent is able to achieve the goal and evade obstacles. A Lyapunov-based stability analysis is used to prove that the target agent regulation error is uniformly ultimately bounded.