Model-based and Model-free Optimal Control of Biomechanical SIP Model

Standing upright is an intricate task achieved through complex coordination of central nervous system (CNS) and a high degree of freedom musculoskeletal system. Biomechanical inverted pendulum models are often employed to model human neuromusculoskeletal system and controllers are designed to analyze such a complex coordination. In this study, we designed a variety of controllers using reinforcement learning (RL) policy iteration and value iteration methods for a single-link inverted pendulum (SIP) model actuated by an ankle joint torque. Policy iteration (PI) and value iteration (VI) algorithms are implemented to design model-based, offline linear quadratic regulator (LQR) controllers and model-free, adaptive, and online LQR controllers. Value function approximation methods are used to design online adaptive controllers where SIP dynamics were assumed unknown. Computer simulations affirm the optimal solution convergence characteristics for both PI and VI methods.