Circuit Driving of RC Scale Cars using Reinforcement Learning

This paper presents control of an RC scale car in a scale circuit using reinforcement learning. Experimental environment has been constructed with 1/27 scale remote controlled car, motion tracking system, and a computer that sends steering and thrust commands to the RC car based on feedback from the motion tracking system. The control consists of two layers. Low-level controller receives a desired velocity vector as a reference and do a basic PI control for thrust and P control for steering. High-level controller is trained by reinforcement learning that receives the car state and outputs the velocity command vector. The state include position, velocity, heading of the RC car, distances to surrounding boundaries of the circuit. The high-level controller takes the form of a recursive neural network, which is trained entirely in virtual environment. The car dynamics in the virtual environment is a bicycle model that includes tire slip force from the literature. With the resulting policy (high-level controller) the RC car successfully completes 10 laps in the actual environment of the circuit without colliding to the boundaries.