Usage Pattern-Driven Dynamic Data Layout Reorganization
| Autor: | Xiaocheng Zou, Houjun Tang, Kesheng Wu, Steve Harenberg, Kristofer E. Bouchard, Scott Klasky, Oliver Rubel, Nagiza F. Samatova, Wenzhao Zhang, Bin Dong, Suren Byna |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2016 |
| Předmět: |
020203 distributed computing
Focus (computing) Data layout Computer science Dynamic data Reading (computer) Selection strategy Locality 02 engineering and technology computer.software_genre 020202 computer hardware & architecture Set (abstract data type) Data access 0202 electrical engineering electronic engineering information engineering Data mining computer |
| Zdroj: | Tang, Houjun; Byna, Suren; Harenberg, Steven; Zou, Xiaocheng; Zhang, Wenzhao; Wu, Kesheng; et al.(2016). Usage Pattern-Driven Dynamic Data Layout Reorganization:. Lawrence Berkeley National Laboratory: Lawrence Berkeley National Laboratory. Retrieved from: http://www.escholarship.org/uc/item/7dq7t3zq CCGrid |
| Popis: | As scientific simulations and experiments move toward extremely large scales and generate massive amounts of data, the data access performance of analytic applications becomes crucial. A mismatch often happens between write and read patterns of data accesses, typically resulting in poor read performance. Data layout reorganization has been used to improve the locality of data accesses. However, current data reorganizations are static and focus on generating a single (or set of) optimized layouts that rely on prior knowledge of exact future access patterns. We propose a framework that dynamically recognizes the data usage patterns, replicates the data of interest in multiple reorganized layouts that would benefit common read patterns, and makes runtime decisions on selecting a favorable layout for a given read pattern. This framework supports reading individual elements and chunks of a multi-dimensional array of variables. Our pattern-driven layout selection strategy achieves multi-fold speedups compared to reading from the original dataset. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|