Reinforcement Learning for Virtually-Coupled Trains Using Q-Learning with Continuous Actions

As an unsupervised and value-based reinforcement learning (RL) algorithm, Q-learning is an effective tool to determine off-policy strategies for corresponding agents which mimic the behaviors of physical plants. In order to further improve the rail-line efficiency, virtual coupling mode has been recognized as an effective development direction, receiving wide concerns from both theoretical research and practical applications. In this paper, a QCA-learning algorithm control strategy with continuous actions (CA), aliased as QCA-learning, is proposed for virtually-coupled trains, to maintain steady tracking distance, by discretizing the train operation state and adjusting the action change rate, speed error and distance as the reward function. Therein, an
$\varepsilon$
-greedy strategy, with a greedy function decreasing with the number of learning, is designed to reduce the invalid exploration of the algorithm and improve the learning efficiency. Finally, simulation results by collected data from G1# train, running from Beijingnan to Jinanxi railway stations, are presented to demonstrate that the QCA-learning algorithm performs better speed and distance convergences in comparison to traditional strategy and guarantee satisfactory performance regarding to steady desired tracking distance among virtually-coupled trains.