RLTiering: A Cost-Driven Auto-Tiering System for Two-Tier Cloud Storage Using Deep Reinforcement Learning

The cloud storage boom has prompted providers to offer two storage tiers, i.e., hot and cold tiers, which are respectively purpose-built to provide the lowest cost for frequent and infrequent access patterns. However, for cloud users, it is non-trivial to determine cost-effective tiers because it is hard to obtain future access patterns in advance and is difficult to predict them exactly. The lack of future information poses a risk of increasing costs instead of saving costs. This is not the only challenge encountered when it comes to cost optimization. In this article, we take Amazon S3 as an example to analyze the pricing of two-tier cloud storage and derive several major challenges faced by cost optimization. Then, assuming a priori knowledge of future access patterns, we propose an optimal offline algorithm based on dynamic programming to determine cost-effective tiers for each time slot. Further, to handle online workload arrivals, we formulate the problem using Markov decision processes and propose RLTiering based on deep reinforcement learning. Eventually, the cost performance of RLTiering is evaluated based on real-world traces and prevalent Amazon S3 pricing, and the results show that it achieves significant cost-savings.