Adaptive Online Runtime Prediction to Improve HPC Applications Latency in Cloud

We propose adaptive online application runtime prediction methods to improve application latency in clouds. Scheduling algorithms are highly sensitive to the value of application runtime. %Currently available schedulers rely on inaccurate user predictions as well as inaccurate over the shelf forecasting tools. Predicting the application runtime values in the cloud is challenging because of highly dynamic nature of application runtime values, the requirement of real-time prediction and scarcity of training data. To provide an accurate and fast online prediction model, we first analyze available HPC traces and design hybrid models from adaptive online generative models known as State Space Models. We show how State Space Models are generalizations of widely used methods for application runtime prediction in HPC clusters including exponential smoothing and recursive least square. To overcome high variance of application runtimes in the cloud, our proposed methods facilitate on-the-fly automated model selection and model switching. Our machine learning prediction results on HPC application traces show that our adaptive online prediction models predict the runtimes 33% to 80% more accurately than existing prediction approaches. Besides, our extensive trace-based scheduling simulations show that our predicted runtimes improve the performance (wait time) by 25%.