Learning Aerial Docking via Offline-to-Online Reinforcement Learning

In this study, we explore the intricate task of executing a docking operation between two quadrotors, utilizing a blend of offline and online reinforcement learning techniques. This multifaceted task is fraught with the catastrophic forgetting problem due to its multiple objectives: it necessitates precise control over the trajectory of the quadrotors, ensures their stability during the docking process, and maintains stable control post-docking. The complexity of these objectives presents significant challenges when applying reinforcement learning directly, often leading to unsuccessful outcomes. To navigate these challenges, we initially developed a rule-based expert controller and amassed a substantial dataset. Subsequently, we employed offline reinforcement learning to train a guided policy, which was then fine-tuned using online reinforcement learning. This approach effectively addressed the out-of-distribution issues typically encountered in online reinforcement learning with guided policies. Notably, our methodology significantly enhanced the success rate of the expert strategy, boosting it from 40% to an impressive 95%
¹
.