Design of attitude and path tracking controllers for quad-rotor robots using reinforcement learning

There is a lot of interest in using quad-rotor helicopters as Miniature Aerial Vehicles (MAVs) due to their simple mechanical construction and straightforward propulsion system. However, since these vehicles are highly unstable nonlinear dynamical systems, a suitable control system is required for their attitude stabilization and navigation. This article presents a simulation environment for the design and evaluation of attitude stabilization and path tracking controllers for quad-rotor aerial robots using Reinforcement Learning (RL). Firstly, the nonlinear mathematical model for a commercial X3D-BL quad-rotor robot from Ascending Technologies is introduced. The attitude stabilization and path tracking controllers for the quad-rotor robot are formulated. It is shown how the parameters of the controllers can be adjusted using a RL algorithm called Learning Automata. Next, the proposed simulation topology is presented and its main features are discussed. It employs 2 host computers where one host executes the control loops and the reinforcement learning algorithm using MATLAB/SIMULINK. The other host runs the quad-rotor robot model using the X-Plane Flight Simulator. The two hosts communicate using UDP (User Datagram Protocol) over a standard Ethernet wired network. Finally, some simulation cases are presented and the controllers adjusted by the RL algorithm are evaluated.

1. Introduction

Miniature Aerial Vehicle's have attracted the attention of many researchers in the past few years. This is due to the broad range of military applications like surveillance, border patrolling, mine detection, crowd control, aerial delivery of payload, and civilian purposes like disaster management during floods, earthquakes, fire, intervention in hostile environment, exploration of high buildings or contaminated areas, rescue missions, disaster relief support, aerial photography, television and cinema shootings, inspection of transmission lines and oil pipelines [1]. In this work we discuss a type of rotary-wing Unmanned Mini Aerial Vehicle, known as quad-rotor. Quad-rotor consists of four independent propellers attached at each corner of a cross frame, see Figure 1. One advantage of the quad-rotor compared to a conventional helicopter is the simplified rotor mechanics. It has vertical take-off and landing capabilities which gives it higher maneuverability and hovering capabilities. Quad-rotor robots exhibit a number of important physical effects such as aerodynamic effects, gravity effect, ground effect, gyroscopic effect and friction, etc. Due to these effects it is difficult to design a good control for quad-rotors [21].