Reinforcement learning-based UAV Swarm motion planning for field monitoring

There has been a rapid increase in the utilization of Unmanned Aerial Vehicles (UAVs) in commercial and industrial sectors due to their low costs and small size. However, in this type of system, it is necessary to consider the problem of computing or identifying optimal paths for each UAV. Finding an efficient solution to this problem would allow the autonomous control of UAVs thus leading to successful completion of assigned tasks within specified time limits. The objective is to develop a reinforcement learning system that is capable of finding optimal trajectories for each UAV agent in the swarm deployed for the task of field prospecting. The reinforcement learning algorithms stand out for their ability to learn through trial and error mechanisms and allow the agent to adapt itself according to the surrounding environment. The reason behind identifying optimal trajectories is to get maximum coverage of an overflight region regardless of the number of UAV agents in the swarm and the size of the map. In this kind of problem, it is not required to specify any target or to have any prior knowledge of the flight environment except the given map. In this work, experiments have been conducted to determine whether it is optimal to use an on-policy SARSA or off-policy Q-learning-based approach for computing the trajectories of UAV agents in the swarm. The results presented in the paper manifest that the performance of both algorithms is comparable. However, in small-sized flight maps Q-learning based path planning scheme converges faster and provides more map coverage. In contrast, better performance is achieved with SARSA based trajectory planning method in large-sized maps representing flight environments.