Conservative Q-Learning Based Furnace Temperature Control in Municipal Solid Waste Incineration Process

The nonlinear, time-varying, and lagging nature of the municipal solid waste incineration (MSWI) process presents challenges for ensuring controller safety. While offline reinforcement learning (RL) can ensure safety in furnace temperature (FT) control, its performance is hindered by extrapolation errors, making it unsuitable for direct application in the incineration environment. To address this, we propose a conservative Q-learning-based furnace temperature control strategy (CQL-FTC). This strategy involves two stages: online sampling and offline training. During the online sampling stage, the agent interacts with the environment to collect samples, building an experience replay buffer (ERB) and performing pretraining. In the offline training stage, we introduce the CQL method, adding constraint terms to the traditional Bellman equation to minimize extrapolation errors. After offline training, the agent is directly applied to the FT setpoint control in the incineration process. Simulation results using the actual MSWI process dataset demonstrate the effectiveness of the proposed method in complex industrial environments.