Reinforcement Learning Compensation based PD Control for a Double Inverted Pendulum

In this paper, we present a Control Algorithm based on Reinforcement Learning for a double inverted pendulum on a cart. By implementing the Q-Learning techniques in the PD control scheme, the second pendulum (top pendulum) is enabled to improve its performance. In a first step, Q-Learning is used so that the control can balance the second pendulum towards its inverted vertical position, while the first pendulum has no restrictions on its movement and also the car remains in a range of ±1 meter in its displacement. In a second step, we combine hybrid techniques of Q-Learning and PD control, in a system that has had changes in its parameters and in its initial conditions. Then, with the hybrid control, we obtain better results than using the controllers individually. Finally, the simulation results show the effectiveness of the proposed controller.