Compression Ratio Modeling and Estimation across Error Bounds for Lossy Compression
| Autor: | Weiming He, Huizhang Luo, Jinzhen Wang, Tong Liu, Qing Liu, Xubin He |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
020203 distributed computing
Computer science Data_CODINGANDINFORMATIONTHEORY 02 engineering and technology Lossy compression Reduction (complexity) Computational Theory and Mathematics Computer engineering Hardware and Architecture Compression (functional analysis) Signal Processing Compression ratio 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Volume (compression) |
| Zdroj: | IEEE Transactions on Parallel and Distributed Systems. 31:1621-1635 |
| ISSN: | 2161-9883 1045-9219 |
| Popis: | Scientific simulations on high-performance computing (HPC) systems generate vast amounts of floating-point data that need to be reduced in order to lower the storage and I/O cost. Lossy compressors trade data accuracy for reduction performance and have been demonstrated to be effective in reducing data volume. However, a key hurdle to wide adoption of lossy compressors is that the trade-off between data accuracy and compression performance, particularly the compression ratio, is not well understood. Consequently, domain scientists often need to exhaust many possible error bounds before they can figure out an appropriate setup. The current practice of using lossy compressors to reduce data volume is, therefore, through trial and error, which is not efficient for large datasets which take a tremendous amount of computational resources to compress. This paper aims to analyze and estimate the compression performance of lossy compressors on HPC datasets. In particular, we predict the compression ratios of two modern lossy compressors that achieve superior performance, SZ and ZFP, on HPC scientific datasets at various error bounds, based upon the compressors’ intrinsic metrics collected under a given base error bound. We evaluate the estimation scheme using twenty real HPC datasets and the results confirm the effectiveness of our approach. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|