A Latency Aware and Dynamic Caching Model for Heterogeneous Datalake Environments

The heterogeneous and multi-structured data within datalakes environment stored at distributed geographical locations raises the difficulty of user query processing. Caching is a technique that can help reduce the latency by facilitating the required web pages within cache memory for faster query processing. The dynamic demands of heterogeneous web content are entangled with critical challenges of adjusting cache size dynamically and scaling processing requirements with elastic resource capacity. However, the existing caching schemes based on duration of web page stay within cache are insufficient to provision dynamic caching of heterogeneous web content automatically. In this context, this paper proposes a novel dynamic and self-adaptive cache management model named latency aware dynamic caching (LAD-Caching). It incorporates deep learning capability of Long Short-term Memory (LSTM) algorithm for proactive estimation of caching contents with an automated cache admission and eviction scheme. The innovative idea of engaging two different LSTM-based units for dynamic cache size prediction and cache content eviction is proposed for the first time which captures all-inclusive dynamic cache management in diverse datalake environment. The simulation and performance evaluation of the proposed LAD-Caching using a benchmark dataset confirms its efficiency in terms of reduced average data access time up to 100.826 nsec as compared with optimal case and minimizing average delay up to 99% over without LAD-Caching. Further, the number of cache hits are improved up to 52.7% and 51.2% over existing caching.