Inter-UAV Collision Avoidance using Deep-Q-Learning in Flocking Environment

In the past twenty years, Unmanned Aerial Vehicles (UAVs) have demonstrated their ability in supporting both military and civilian applications. It has proved itself useful in tasks which are dangerous or too costly with usual methods. Certain applications also contains tasks which can be processed faster by using multiple UAVs that work together and perform smaller tasks which can be further combined to get the intended result. However, to achieve this, we need autonomous coordination among the multi UAVs. Multiple UAVs flying as a swarm, incorporates a pattern or a shape that has to be maintained through out the flight. But there can be cases where the pattern or shape of swarm has to be changed. Under such circumstances, collision has to be avoided between the UAVs as they travel to their new position forming a new shape. We have proposed an algorithm which helps in finding optimal goal positions for the UAVs to flock and to find the trajectory which supports collision free movement to its assigned position. We also ensure that the total distance travelled by individual UAV is minimized. The optimal goal position assignment is done using Hungarian Algorithm. Deep Q Learning method is used to find the optimal flight parameters for a collision free trajectory for the UAVs to reach its goal. Results from simulation show that the algorithm is sufficiently fast for practical applications as optimal assignments and flight parameters were computed for 50 UAVs in less than 1s.