Immersion and Invariance-based Adaptive Control for Quadrotor Transportation Systems Using Deep Reinforcement Learning | IEEE Conference Publication | IEEE Xplore

Immersion and Invariance-based Adaptive Control for Quadrotor Transportation Systems Using Deep Reinforcement Learning


Abstract:

Quadrotor unmanned aerial vehicles (UAVs) are playing important roles in cargo transportation, especially in complex and unknown environments. However, the underactuated ...Show More

Abstract:

Quadrotor unmanned aerial vehicles (UAVs) are playing important roles in cargo transportation, especially in complex and unknown environments. However, the underactuated nature of the quadrotor UAV transportation systems makes it difficult for control schemes design. Considering system parameter uncertainties, a novel nonlinear adaptive control law is designed via Immersion and Invariance (I&I) method. Besides, in order to pursue faster convergence and stronger robustness, a novel control structure based on deep reinforcement learning is presented. The RL agent, which is designed through twin delayed deep deterministic policy gradient (TD3) algorithm, generates optimal control gains for the I&I adaptive controller in real-time. The stability of the closed-loop system are guaranteed through Lyapunov technique and LaSalle’s invariance theorem. Finally, by comparing with the classical control scheme, the superior performance and stronger robustness of the proposed controller are validated through numerous simulations.
Date of Conference: 09-11 July 2022
Date Added to IEEE Xplore: 29 November 2022
ISBN Information:
Conference Location: Guilin, China

Funding Agency:

References is not available for this document.

I. Introduction

In the past decades, quadrotor UAVs have become popular due to their advantages of maneuverability [1] and the vertical take-off and landing (VTOL) capability [2], [3]. These features make the quadrotors ideal choices for cargo delivery by air [4]. Among the basic ways of the quadrotor transportation [5], cable-suspended load transportation has received increasing attention recently. However, the control of the quadrotor transportation systems is a much more challenging task compared with single quadrotor control problems due to the dual underactuated property, strong nonlinearity and complex dynamic coupling [6], hence it is of great necessity to design efficient control schemes for mission safety.

Select All
1.
S. Gupte, P.I.T. Mohandas and J.M. Conrad, "A Survey of Quadrotor Unmanned Aerial Vehicles", pp. 323-336, 2012.
2.
G. Hoffmann, H. Huang, S. Waslander and C. Tomlin, "Quadrotor Helicopter Flight Dynamics and Control: Theory and Experiment", AIAA Guidance Navigation and Control Conference, pp. 1670-1689, 2007.
3.
A. Tayebi and S. McGilvray, "Attitude Stabilization of a VTOL Quadrotor Aircraft", vol. 14, no. 3, pp. 562-571, 2006.
4.
D. K. D. Villa, A. S. Brandao and M. Sarcinelli-Filho, "Load Transportation Using Quadrotors: A Survey of Experimental Results", 2018 International Conference on Unmanned Aircraft Systems, pp. 84-93, 2018.
5.
D. K. Villa, A. S. Brandao and M. Sarcinelli-Filho, "A Survey on Load Transportation Using Multirotor Uavs", Journal of Intelligent Robotic Systems, vol. 98, no. 2, pp. 267-296, 2020.
6.
X. Liang, Y. Fang, N. Sun, H. Lin and X. Zhao, "Adaptive Nonlinear Hierarchical Control for a Rotorcraft Transporting a Cable-Suspended Payload", IEEE Transactions on Systems Man and Cybernetics: Systems, vol. 51, no. 7, pp. 4171-4182, 2021.
7.
J. Potter, W. Singhose and M. Costelloy, "Reducing Swing of Model Helicopter Sling Load Using Input Shaping", 2011 IEEE International Conference on Control and Automation, pp. 348-353, 2011.
8.
I. Palunko, R. Fierro and P. Cruz, "Trajectory Generation for Swingfree Maneuvers of A Quadrotor with Suspended Payload: A Dynamic Programming Approach", 2012 IEEE International Conference on Robotics and Automation, pp. 2691-2697, 2012.
9.
A. Faust, I. Palunko, P. Cruz, R. Fierro and L. Tapia, "Learning Swingfree Trajectories for UAVs with A Suspended Load", 2013 IEEE International Conference on Robotics and Automation, pp. 4902-4909, 2013.
10.
I. H. B. Pizetta, A. S. Brandao and M. Sarcinelli-Filho, "Modelling and Control of A PVTOL Quadrotor Carrying A Suspended Load", 2015 International Conference on Unmanned Aircraft Systems, pp. 444-450, 2015.
11.
S. Yang and B. Xian, "Trajectory Tracking Control Design for the System of A Quadrotor UAV with A Suspended Payload", 2017 36th Chinese Control Conference, pp. 777-782, 2017.
12.
X. Liang, Z. Zhang, H. Yu, Y. Wang, Y. Fang and J. Han, "Antiswing Control for Aerial Transportation of the Suspended Cargo by Dual Quadrotor UAVs", IEEE/ASME Transactions on Mechatronics, 2022.
13.
P. Borja, R. Cisneros and R. Ortega, "A Constructive Procedure for Energy Shaping of Port-Hamiltonian Systems", Automatica, vol. 72, pp. 230-234, 2016.
14.
X. Liang, H. Yu, Z. Zhang, H. Liu, Y. Fang and J. Han, "Unmanned Aerial Transportation System with Flexible Connection between the Quadrotor and the Payload: Modeling Controller Design and Experimental Validation", IEEE Transactions on Industrial Electronics, 2022.
15.
A. Alleyne and J. K. Hedrick, "Nonlinear Adaptive Control of Active Suspensions", IEEE Transactions on Control Systems Technology, vol. 3, no. 1, pp. 94-101, 1995.
16.
A. Astolfi and R. Ortega, "Immersion and Invariance: A New Tool for Stabilization and Adaptive Control of Nonlinear Systems", IEEE Transactions on Automatic Control, vol. 48, no. 4, pp. 590-606, 2003.
17.
J. Hu and H. Zhang, "Immersion and Invariance Based Command-filtered Adaptive Backstepping Control of VTOL Vehicles", Automatica, vol. 49, no. 7, pp. 2160-2167, 2013.
18.
B. Zhao, B. Xian, Y. Zhang and X. Zhang, "Nonlinear Robust Adaptive Tracking Control of a Quadrotor UAV Via Immersion and Invariance Methodology", IEEE Transactions on Industrial Electronics, vol. 62, no. 5, pp. 2891-2902, 2015.
19.
Y. Wang, J. Sun, H. He and C. Sun, "Deterministic Policy Gradient With Integral Compensator for Robust Quadrotor Control", IEEE Transactions on Systems Man and Cybernetics: Systems, vol. 50, no. 10, pp. 3713-3725, 2020.
20.
H. Qin, P. Tan, Z. Chen, M. Sun and Q. Sun, "Deep Reinforcement Learning Based Active Disturbance Rejection Control for Ship Course Control", Neurocomputing, vol. 484, pp. 99-108, 2021.
21.
J. Hwangbo, I. Sa, R. Siegwart and M. Hutter, "Control of a Quadrotor With Reinforcement Learning", IEEE Robotics and Automation Letters, vol. 2, no. 4, pp. 2096-2103, 2017.
22.
S. Fujimoto, H. Hoof and D. Meger, "Addressing Function Approximation Error in Actor-critic Methods", International conference on Machine Learning, pp. 1587-1596, 2018.
23.
H. K. Khalil, Nonlinear Systems, Englewood Cliffs, NJ:Prentice-Hall, 2002.

References

References is not available for this document.