Mediating Data Center Storage Diversity in HPC Applications with FAODEL
| Autor: | Gary J. Templet, Scott Levy, Craig D. Ulmer, Patrick Widener, Todd Kordenbrock |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Service (systems architecture)
business.industry Computer science Distributed computing Data management 020206 networking & telecommunications 02 engineering and technology Supercomputer Data type Workflow Computer data storage Scalability 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Data center business |
| Zdroj: | Lecture Notes in Computer Science ISBN: 9783030343552 ISC Workshops |
| Popis: | Composition of computational science applications into both ad hoc pipelines for analysis of collected or generated data and into well-defined and repeatable workflows is becoming increasingly popular. Meanwhile, dedicated high performance computing storage environments are rapidly becoming more diverse, with both significant amounts of non-volatile memory storage and mature parallel file systems available. At the same time, computational science codes are being coupled to data analysis tools which are not filesystem-oriented. In this paper, we describe how the FAODEL data management service can expose different available data storage options and mediate among them in both application- and FAODEL-directed ways. These capabilities allow applications to exploit their knowledge of the different types of data they may exchange during a workflow execution, and also provide FAODEL with mechanisms to proactively tune data storage behavior when appropriate. We describe the implementation of these capabilities in FAODEL and how they are used by applications, and present preliminary performance results demonstrating the potential benefits of our approach. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|