Reducing network cost of data repair in erasure-coded cross-datacenter storage

Autor: Fangliang Xu, Bao Han, Yijie Wang
Rok vydání: 2020
Předmět:
Zdroj: Future Generation Computer Systems. 102:494-506
ISSN: 0167-739X
DOI: 10.1016/j.future.2019.08.027
Popis: Nowadays, cross-datacenter storage is widely used because of its high disaster-tolerance. Moreover, the erasure code is gradually being applied to cross-datacenter storage since it offers the same reliability as replication with a significant decrease in the amount of storage required. However, the network cost of repairing erasure-coded data in existing cross-datacenter storage (repair cost of cross-datacenter erasure code) is high, which usually results in a long repair time. To reduce the repair time by reducing the repair cost, in this paper, we study both the encoding method and repair method of cross-datacenter erasure code. First, we propose the concept of average weighted locality, which is proportional to the average repair cost of cross-datacenter erasure code. Then, we propose a network environment-adaptive encoding method of cross-datacenter erasure code (NEC). Under different network environments and encoding parameters, NEC can compute the approximate optimal generator matrix and data placement scheme online in a parallel heuristic way to achieve the approximate smallest average weighted locality. After that, we propose a hybrid-structured repair method of cross-datacenter erasure code (HRepair), which can achieve a good tradeoff between repair cost and repair efficiency by constructing a hybrid tree-star transmission topology to organize data transmission in the repair process. Experiments show that: Compared with several state-of-the-art erasure codes, NEC can reduce the repair cost and repair time by 26.8%–40% and 13.9%-37.1%. Besides, HRepair can reduce repair cost and repair time further.
Databáze: OpenAIRE
Externí odkaz: