Heterogeneous Job Allocation Scheduler for Hadoop MapReduce Using Dynamic Grouping Integrated Neighboring Search
| Autor: | Chi-Ting Chen, Albert Y. Zomaya, Ling-Ju Hung, Rajkumar Buyya, Sun-Yuan Hsieh |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Job scheduler
Computer Networks and Communications Computer science business.industry Distributed computing Locality 020206 networking & telecommunications Symmetric multiprocessor system Cloud computing 02 engineering and technology Dynamic priority scheduling computer.software_genre Computer Science Applications Scheduling (computing) Hardware and Architecture 0202 electrical engineering electronic engineering information engineering Information system 020201 artificial intelligence & image processing Data center business computer Software Information Systems |
| Zdroj: | IEEE Transactions on Cloud Computing. 8:193-206 |
| ISSN: | 2372-0018 |
| DOI: | 10.1109/tcc.2017.2748586 |
| Popis: | MapReduce is a crucial framework in the cloud computing architecture, and is implemented by Apache Hadoop and other cloud computing platforms. The resources required for executing jobs in a large data center vary according to the job types. In general, there are two types of jobs, CPU-bound and I/O-bound, which require different resources but run simultaneously in the same cluster. The default job scheduling policy of Hadoop is first-come-first-served and therefore, may cause unbalanced resource utilization. Considering various job workloads, numerous job allocation schedulers were proposed in the literature. However, those schedulers encountered the data locality problem or unreasonable job execution performance. This study proposes a job scheduler based on a dynamic grouping integrated neighboring search strategy, which can balance the resource utilization and improve the performance and data locality in heterogeneous computing environments. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|