Improving a Proportional Integral Controller with Reinforcement Learning on a Throttle Valve Benchmark

This paper presents a learning-based control strategy for non-linear throttle valves with an asymmetric hysteresis leading to a near-optimal controller. We start with a carefully tuned Proportional Integrator (PI) controller and exploit the recent advances in Reinforcement Learning (RL) with Guides to improve the closed-loop behavior by learning from the additional interactions with the valve. We test the proposed control method in various scenarios on a throttle valve benchmark, highlighting the benefits of combining both PI and RL frameworks to improve control performance in non-linear stochastic systems. In all the experimental test cases, the resulting agent has a better sample efficiency than traditional RL agents and outperforms the PI controller.