Dynamic Stabilization of Unstable Limit Cycles in Passive Walking via Deep Reinforcement Learning

In this research, Deep Reinforcement Learning (DRL) is employed to develop an ankle controller for a passive biped walker with round feet. The selected DRL technique is Soft Actor-Critic. A reward function is formulated and tailored to the specific requirements of the RL framework. Notably, this controller leverages passive dynamic Basin of Attraction data as initial conditions for the training episodic resets, which is an inherent characteristic of the model. A custom Gym environment is constructed to facilitate the testing of reinforcement learning algorithms. The findings are validated against a controller based on the Reversible Switching Surface (RSS) technique. The investigation demonstrates that the trained controller exhibits stable and robust gaits, achieving results appropriate to the optimal RSS controller gains without the need for additional sensitivity analysis.