Identifying Latent Reduced Models to Precondition Lossy Compression
| Autor: | Dan Huang, Haitao Yuan, Zhenbo Qiao, Huizhang Luo, MengChu Zhou, Qing Liu, Hong Jiang, Zhenlu Qin, Jinzhen Wang, Jing Bi |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Discrete wavelet transform
020203 distributed computing Computer science Dimensionality reduction 020207 software engineering Data_CODINGANDINFORMATIONTHEORY 02 engineering and technology Lossy compression Reduction (complexity) Compression (functional analysis) Singular value decomposition 0202 electrical engineering electronic engineering information engineering Algorithm Volume (compression) |
| Zdroj: | IPDPS |
| Popis: | With the high volume and velocity of scientific data produced on high-performance computing systems, it has become increasingly critical to improve the compression performance. Leveraging the general tolerance of reduced accuracy in applications, lossy compressors can achieve much higher compression ratios with a user-prescribed error bound. However, they are still far from satisfying the reduction requirements from applications. In this paper, we propose and evaluate the idea that data need to be preconditioned prior to compression, such that they can better match the design philosophies of a compressor. In particular, we aim to identify a reduced model that can be utilized to transform the original data to a more compressible form. We begin with a case study of Heat3d as a proof of concept, in which we demonstrate that a reduced model can indeed reside in the full model output, and can be utilized to improve compression ratios. We further explore more general dimension reduction techniques to extract the reduced model, including principal component analysis, singular value decomposition, and discrete wavelet transform. After preconditioning, the reduced model in conjunction with delta is stored, which results in higher compression ratios. We evaluate the reduced models on nine scientific datasets, and the results show the effectiveness of our approaches. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|