| Autor: |
Jinzhen Wang, Huizhang Luo, Zhenbo Qiao, Hong Jiang, Qing Liu, Mengxiao Wang |
| Rok vydání: |
2018 |
| Předmět: |
|
| Zdroj: |
IEEE Letters of the Computer Society. 1:5-8 |
| ISSN: |
2573-9689 |
| Popis: |
High-performance computing (HPC) applications generate large amounts of floating-point data that need to be stored and analyzed efficiently to extract the insights and advance knowledge discovery. With the growing disparities between compute and I/O, optimizing the storage stack alone may not suffice to cure the I/O problem. There has been a strong push in the HPC communities to perform data reduction before data is transmitted to storage in order to lower the I/O cost. However, as of now, neither lossless nor lossy compressors can achieve the adequate reduction ratio that is desired by applications. This paper proposes DuoModel, a new approach that leverages the similarity between the full and reduced application models, and further improve the data reduction ratio. DouModel further improves the compression ratio of state-of-the-art compressors via compressing the differences (termed as delta) between the data products of the two models. For data analytics, the high fidelity data can be re-computed by launching the reduced model and applying the compressed delta. Our evaluations confirm that DuoModel can further push the limit of data reduction while the high fidelity of data is maintained. |
| Databáze: |
OpenAIRE |
| Externí odkaz: |
|
