An Improved Replica Placement Policy for Hadoop Distributed File System Running on Cloud Platforms
| Autor: | Ibrahim Adel Ibrahim, Mostafa A. Bassiouni, Wei Dai |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
business.industry
Computer science Replica Distributed computing Big data 020206 networking & telecommunications Cloud computing 02 engineering and technology 020202 computer hardware & architecture Homogeneous 0202 electrical engineering electronic engineering information engineering Cluster (physics) Completion time business Distributed File System |
| Zdroj: | CSCloud |
| DOI: | 10.1109/cscloud.2017.65 |
| Popis: | Load balance is a crucial issue for data-intensive computing on cloud platforms, because a load balanced cluster can significantly improve the completion time of data-intensive jobs. In this paper, we present an improved replica placement policy for Hadoop Distributed File System (HDFS), which is specifically designed for heterogeneous clusters. The HDFS replica placement policy cannot generate balanced replica assignment, and hence has to rely on a load balance utility to balance the load among cluster nodes. In contrast, our proposed policy can generate perfectly even replica assignment, and also achieve load balance among cluster nodes in any heterogeneous or homogeneous environments without the running of the load balance utility. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|