Development of a framework to reduce overhead on database engine through data distribution

Software driven solutions are limited to the amount of memory size and storage capacity, but the sizes of databases are increasing every day. Hence, now a day, handling data and accessing it in an acceptable time is one of the biggest challenges especially in a large database system. In a database, the records can be categorized according to the access frequencies; some records are very frequently accessed (hot data), some records are hardly accessed (cold data) and other records accessed occasionally (warm data). In a conventional database we keep all hot, warm and cold records in a single database. In case of record access (query, update etc.) a query might takes longer time even if a good data accessing algorithm (clustering/mining) incorporate with the database. Thus categorizing of the data set, i. e. clustering in terms of access frequency may improve data accessibility. In this paper, we are proposing a data clustering mechanism based on data access frequency. Finally, the expected result shows how and why data accessibility time should outperform other available data clustering techniques.

I. Introduction

During the last two decades, Relational Database Management System (RDBM) has been established as the technology and giving us the capabilities to handle databases up to terabytes [1]. Our modern information technology allow us to produce even more powerful computers every year, which makes it possible to collect, store, transfer, and combine huge amounts of data at very low costs. Thus, companies and scientific research institutions can afford to build up large archives of documents and data like numbers, tables, images, and sounds. However, querying and exploiting data in an intelligent way from a gigantic database and represent it in a structured way is turning out to be fairly difficult. It will be almost impossible to handle or access this large amount of data if database operation is not performed in an optimized way or data is not properly clustered. When the data access routines of that application are not optimized, there is a 90% possibility is that a database based application performs slowly [2]. So far, scientific works have been done on reducing overhead on database related to memory uses and storage uses. And more scientific works have been performed on clustering data base from different angle. However, clustering data based on hot/warm/cold (frequently accessed, least rarely accessed and least accessed) is somewhat missing.