Interference and locality-aware task scheduling for MapReduce applications in virtual clusters
| Autor: | Cheng-Zhong Xu, Xiangping Bu, Jia Rao |
|---|---|
| Rok vydání: | 2013 |
| Předmět: |
0209 industrial biotechnology
Computer science business.industry Distributed computing 020208 electrical & electronic engineering Locality Cloud computing 02 engineering and technology Dynamic priority scheduling computer.software_genre Virtualization Fair-share scheduling Scheduling (computing) 020901 industrial engineering & automation Virtual machine Two-level scheduling 0202 electrical engineering electronic engineering information engineering business computer |
| Zdroj: | HPDC |
| DOI: | 10.1145/2462902.2462904 |
| Popis: | MapReduce emerges as an important distributed programming paradigm for large-scale applications. Running MapReduce applications in clouds presents an attractive usage model for enterprises. In a virtual MapReduce cluster, the interference between virtual machines (VMs) causes performance degradation of map and reduce tasks and renders existing data locality-aware task scheduling policy, like delay scheduling, no longer effective. On the other hand, virtualization offers an extra opportunity of data locality for co-hosted VMs. In this paper, we present a task scheduling strategy to mitigate interference and meanwhile preserving task data locality for MapReduce applications. The strategy includes an interference-aware scheduling policy, based on a task performance prediction model, and an adaptive delay scheduling algorithm for data locality improvement. We implement the interference and locality-aware (ILA) scheduling strategy in a virtual MapReduce framework. We evaluated its effectiveness and efficiency on a 72-node Xen-based virtual cluster. Experimental results with 10 representative CPU and IO-intensive applications show that ILA is able to achieve a speedup of 1.5 to 6.5 times for individual jobs and yield an improvement of up to 1.9 times in system throughput in comparison with four other MapReduce schedulers. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|