A Dynamic Chunking Algorithm Approach for Data Deduplication

As a commonly used data reduction technology in storage systems, deduplication technology is used to eliminate redundant data in data and ensure that the deduplicated data can be recovered accurately, which can effectively improve the utilization rate of storage space and bandwidth. As the core of data duplication technology, the data chunking algorithm can directly affect the efficiency of deduplication. In this paper, Dynamic Chunking Algorithm (DCA) is proposed to optimize the fixed long chunking algorithm and CDC algorithm, and experiments show that DCA achieves good results in dedup ratio, chunking speed and metadata suppression compared with the above two algorithms.