Compression for Time Series Databases Using Independent and Principal Component Analysis

Reliable management of modern cloud computing infrastructures is unrealizable without monitoring and analysis of a huge number of system indicators (metrics) as time series data stored in big databases. Efficient storage and processing of collected historical data from all "objects" of those infrastructures are technology challenges for this Big Data application. We propose a data compression framework for databases of time series that applies correlation content of the data set. Specifically, the fundamental statistical concepts of independent component analysis (ICA) and principal component analysis (PCA) are employed to demonstrate the viability of the approach. We experimentally show significant compression rates for real data sets from IT systems.

I. Introduction

Modern cloud computing services are enabled by huge, hierarchically layered, and deeply virtualized infrastructures with complex interrelations between constituent components (Colman-Meixner et al [1]). Real-time management of those giant infrastructures strongly relies on monitoring and measuring every possible performance or capacity indicator of the system and represents a big data science application challenge.