Can I/O Variability Be Reduced on QoS-Less HPC Storage Systems?
| Autor: | Norbert Podhorszki, Jeremy Logan, George Ostrouchov, Qing Liu, Jong Choi, Xubin He, Matthew Wolf, Dan Huang, Scott Klasky |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Input/output
business.industry Computer science Quality of service Distributed computing 02 engineering and technology Bandwidth throttling 020202 computer hardware & architecture Theoretical Computer Science Computational Theory and Mathematics Hardware and Architecture Computer data storage 0202 electrical engineering electronic engineering information engineering business Software |
| Zdroj: | IEEE Transactions on Computers. 68:631-645 |
| ISSN: | 2326-3814 0018-9340 |
| Popis: | For a production high-performance computing (HPC) system, where storage devices are shared between multiple applications and managed in a best effort manner, I/O contention is often a major problem. In this paper, we propose a balanced messaging-based re-routing in conjunction with throttling at the middleware level. This work tackles two key challenges that have not been fully resolved in the past: whether I/O variability can be reduced on a QoS-less HPC storage system, and how to design a runtime scheduling system that can scale up to a large amount of cores. The proposed scheme uses a two-level messaging system to re-route I/O requests to a less congested storage location so that write performance is improved, while limiting the impact on read by throttling re-routing. An analytical model is derived to guide the setup of optimal throttling factor. We thoroughly analyze the virtual messaging layer overhead and explore whether the in-transit buffering is effective in managing I/O variability. Contrary to the intuition, in-transit buffer cannot completely solve the problem. It can reduce the absolute variability but not the relative variability. The proposed scheme is verified against a synthetic benchmark as well as being used by production applications. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|