On-the-Fly Automated Storage Tiering with Caching and both Proactive and Observational Migration

A hybrid storage system consisting of SSDs and HDDs improves the performance of user IOs by moving IO concentrated areas from the HDDs to the SSDs. It is important for the system to identify such areas and move them to the SSDs immediately. We studied the IO concentrations of operational storage servers and found one type that they continue for up to an hour within a narrow region of storage volume. The narrow region is either the same continued area or the area which moves to the neighboring after 3 minutes in average had passed. Moreover, it appeared at random logical block addresses (LBAs). We decide to call such IO concentrations Wandering IO Concentrated Areas (WIOCAs). To handle these concentrations, we developed an on-the-fly automated storage tiering (OTF-AST) with a caching system and both proactive and observational migration features. This method dynamically migrates the WIOCAs to the SSD of the OTF-AST and hands over the other IO concentrations to cache. The proactive migration feature migrates candidate WIOCAs to an area that will likely become one in the near future. On the other hand, the observational migration feature determines the HDD overhead by checking the user response time and migrates the sub-LUNs that were but are no longer parts of a WIOCA from the SSD to the HDD when the HDD overhead is low. Experimental results showed that our method was 113% faster than Facebook FlashCache on one of the Microsoft Research Cambridge workloads.