Optimization of the Ice Storage Air Conditioning System Operation Based on Deep Reinforcement Learning

With the intention of obtaining the room temperature and economic cost control strategy of an ice storage air conditioning system in a small office building in Shanghai, the ice storage air conditioning system is established in this paper as a Markov decision process model and deep reinforcement learning algorithms are adopted to optimize its operation. In order to avoid the problem of over-dimension and over-estimation of value function caused by reinforcement learning, the DDQN (Double Deep Q-Network) algorithm with dual neural network structure is adopted to optimize the operation of ice storage air-conditioning system. Aiming at overcoming the shortcoming of slow convergence of DDQN algorithm, the action space of DDQN is taken into consideration in this paper. Firstly, the appropriate action set is selected according to the convergence speed of different actions. Secondly, the exponential function is addressed in the reward function. Based on the exponential function, the reward function can adjust the penalty value according to the difference between the expected room temperature and the actual room temperature, thus speeding up the convergence of the DDQN algorithm. Finally, Python is adopted to model and simulate buildings and ice storage air conditioning systems. The simulation results show that the operating cost and the proportion of uncomfortable time are both reduced by DDQN control method proposed in this paper with better control performance.

1 Introduction

With the improvement of people’s living standards, urban construction has developed rapidly, and an increasing number of public buildings leads to the energy consumption of buildings increasing. The operating energy consumption of air-conditioning systems accounts for about half of the total energy consumption of buildings ^[1]. In summer, the energy consumption of the air-conditioning system in hot areas can even reach 70% of the total energy consumption of the building, and the peak load will increase accordingly, widening the peak-to-valley difference between the grid and triggering a profound contradiction between supply and demand. At present, ice storage air conditioning technology can effectively and reasonably solve the problem of excessive load caused by the peak power consumption air conditioning system. How to achieve a balance between indoor personnel comfort and operating costs is the key to the operation optimization of the ice storage air conditioning system^[2].