A Self-Matching Sliding Block Algorithm Applied to Deduplication in Distributed Storage System

Autor:	Sihan Qing, Ying Huo, Chuiyi Xie, Shoushan Luo, Lingli Hu
Rok vydání:	2016
Předmět:	Matching (graph theory) Computer science business.industry Node (networking) Checksum Computer data storage Fingerprint (computing) Distributed data store Data_FILES Data deduplication business Algorithm Block (data storage)
Zdroj:	Information and Communications Security ISBN: 9783319298139 ICICS
Popis:	The deduplication technology can significantly reduce the amount of storage in data centers, thus to save network bandwidth and decrease the cost of construction and maintenance. Having inspired by the sliding block method of the Sliding Block (SB) algorithm and independent block-dividing thought of the Content Defined Chunking (CDC) algorithm, a Self-Matching Sliding Block (SMSB) algorithm for deduplication is proposed. Via communication with metadata node, the storage system client builds a matching table in local memory that contains fingerprint and checksum, based on the matching table to realize sliding block self-matching so as to detect the duplicate blocks. The experimental results show that the deduplication rate and the disk space utilization rate of SMSB algorithm is respectively 2.03 times and 1.28 times of the CDC algorithm and that the data processing speed is 0.83 times of the CDC algorithm. The SMSB algorithm is suitable for distributed storage system.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_________::bb15743ceed880f19290d45d7ef81e50 https://doi.org/10.1007/978-3-319-29814-6_34 Zobrazit plný text záznamu