Parallel Implementation of Lossy Data Compression for Temporal Data Sets
| Autor: | Ankit Agrawal, Seung Woo Son, Christoph Federrath, Alok Choudhary, William Hendrix, Zheng Yuan, Wei-keng Liao |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
FOS: Computer and information sciences
Computer science 010103 numerical & computational mathematics Lossy compression 01 natural sciences Computational science Temporal database Dimension (vector space) Computer Science - Distributed Parallel and Cluster Computing Transfer (computing) 0103 physical sciences Table (database) Distributed Parallel and Cluster Computing (cs.DC) 0101 mathematics 010303 astronomy & astrophysics Data compression |
| Zdroj: | HiPC |
| DOI: | 10.48550/arxiv.1703.02438 |
| Popis: | Many scientific data sets contain temporal dimensions. These are the data storing information at the same spatial location but different time stamps. Some of the biggest temporal datasets are produced by parallel computing applications such as simulations of climate change and fluid dynamics. Temporal datasets can be very large and cost a huge amount of time to transfer among storage locations. Using data compression techniques, files can be transferred faster and save storage space. NUMARCK is a lossy data compression algorithm for temporal data sets that can learn emerging distributions of element-wise change ratios along the temporal dimension and encodes them into an index table to be concisely represented. This paper presents a parallel implementation of NUMARCK. Evaluated with six data sets obtained from climate and astrophysics simulations, parallel NUMARCK achieved scalable speedups of up to 8788 when running 12800 MPI processes on a parallel computer. We also compare the compression ratios against two lossy data compression algorithms, ISABELA and ZFP. The results show that NUMARCK achieved higher compression ratio than ISABELA and ZFP. Comment: 10 pages, HiPC 2016 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|