Characterizing I/O and Storage Activity on the K Computer for Post-Processing Purposes
| Autor: | Fumiyoshi Shoji, Jorji Nonaka, Mario A. R. Dantas, Kenji Ono, Eduardo C. Inacio |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Input/output
Database business.industry Computer Applications Computer science Data management Volume (computing) 02 engineering and technology computer.software_genre Supercomputer 020202 computer hardware & architecture Visualization Data modeling Task (computing) Data visualization 020204 information systems 0202 electrical engineering electronic engineering information engineering business computer |
| Zdroj: | ISCC |
| DOI: | 10.1109/iscc.2018.8538488 |
| Popis: | An increasing volume of data is produced by computational science applications executing on flagship-class supercomputers, such as the K computer. Most of these huge datasets would later pass through post-processing for visualization and analysis in order to derive meaningful information. Particular characteristics of the computing environment, application, and the dataset itself, can make efficiently exploring the performance capabilities of large-scale storage systems supporting these supercomputer a challenging task. This paper presents a characterization of the I/O and storage activity of jobs executed on the K computer focusing on post-processing purposes, based upon nine months of production operation recorded. Results demonstrate the intensive data demand of K computer applications, both in terms of volume of file I/O carried out during job execution, amount of data staged-in and staged-out, and number of files produced per job. These aspects shed light on challenges and opportunities for specialized data management libraries for posthoc data visualization and analysis. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|