PPO Based Reinforcement Learning Horizontal Pods Auto-Scaling-A Case Study in Simulation Environment

Microservice has become a state-of-art architecture in the software engineering domain. In the big data scenario, the data can be encapsulated by microservice, providing independent service or orchestrating the business procedure. Much research, such as threshold-based, rule-based, and even deep learning-based resource scaling, has been proposed to tackle this problem. However, the model trained on data like log or monitor index limited the intended dynamic resource adjusting, which caused high latency during adjusting. Moreover, the computation complexity and the data latency can still be a hindrance to the problem-solving. This paper proposed a reinforcement learning-based horizontal pods auto-scaling algorithm to study and respond in low latency. Furthermore, a case study to validate the effectiveness was carried out in a simulation environment. The results show that the proposed algorithm can converge after 0.3 million steps, and three groups of experiments were carried out, which shows that the trained reinforcement learning model can adjust the replica number in real-time, which indicates that the proposed algorithm can be applied in a scenario with low resource consumption.