A Prediction based Autoscaling in Serverless Computing

Nowadays, serverless computing has risen rapidly as an emerging cloud computing paradigm in recent years. Auto-scaling is a crucial enabler for adapting to workload changes by scaling instances corresponding to the number of incoming requests. Recently, a popular Kubernetes-based platform for managing serverless workloads known as Knative has been proposed. Its scaling algorithm increases and decreases resources based on configured parameter values such as a maximum number of requests that can be processed in parallel per pod. However, users have no idea about the appropriate parameter values, while those predefined values can strongly impact the performance. The negative influence can cause response time increase and throughput reduction if those are overprovisioning or under-provisioning. Besides, Knative uses a moving average method to calculate the number of pods based on past data but cannot reflect the future workload trend, leading to the delay effect. In this paper, we focus on maximizing performance with the lowest resource utilization and optimizing response time to satisfy the quality of service (QoS) requirements so that it is content with users' experience while users only must pay the least cost. To do that, our proposed approach first finds effective scaling policies per pod to maximize the performance with the acceptable latency. Then we apply the forecasting model Bi-LSTM to optimize the number of pods calculation and make Knative more adaptive to the workload changes to reduce the delay time. Our preliminary experiments show that our proposed model can significantly improve performance with an effective scaling policy and an adaptive pod calculation method compared to the default Knative auto-scaling scheme.