Multi-engine Analytics with IReS
| Autor: | Katerina Doka, Nikolaos Papailiou, Victor Giannakouris, Nectarios Koziris, Dimitrios Tsoumakos, Ioannis Mytilinis |
|---|---|
| Rok vydání: | 2019 |
| Předmět: | |
| Zdroj: | Real-Time Business Intelligence and Analytics ISBN: 9783030241230 BIRTE (Revised Selected Papers) |
| Popis: | We present IReS, the Intelligent Resource Scheduler that is able to abstractly describe, optimize and execute any batch analytics workflow with respect to a multi-objective policy. Relying on cost and performance models of the required tasks over the available platforms, IReS allocates distinct workflow parts to the most advantageous execution and/or storage engine among the available ones and decides on the exact amount of resources provisioned. Moreover, IReS efficiently adapts to the current cluster/engine conditions and recovers from failures by effectively monitoring the workflow execution in real-time. Our current prototype has been tested in a plethora of business driven and synthetic workflows, proving its potential of yielding significant gains in cost and performance compared to statically scheduled, single-engine executions. IReS incurs only marginal overhead to the workflow execution performance, managing to discover an approximate pareto-optimal set of execution plans within a few seconds. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|