Imperfect Maintenance Optimization of Multi-State Rolling Stocks Based on Deep Reinforcement Learning

Developing an effective maintenance schedule for the rolling stocks has always been a critical issue of the railway companies. Currently, the Chinese railway companies still schedule the maintenance activities periodically according to the miles the rolling stocks traveled, which cause serious over-maintenance for the purpose of satisfying high reliability requirement. In this paper, we consider an imperfect maintenance optimization problem for multiple rolling stocks with stochastic maintenance time and operating conditions. The rolling stocks are modeled as the multi-state systems to characterize the degradation process. Meanwhile, the operating condition and the current location of the rolling stocks are also taken into consideration. The transition dynamics of the degradation process depends on the operating conditions. The optimization problem is formulated as a continuous-time Markov decision process and the objective is to maximize the total discounted reward related to the operating profit and the cost due to the maintenance, replacement and transportation in an infinite planning horizon. A deep reinforcement learning algorithm is developed to obtaining the optimal maintenance policy of the rolling stocks. A numerical experiment is given to demonstrate the advantage of the algorithm to improve the maintenance schedule of the rolling stocks.