Efficient and Portable Distribution Modeling for Large-Scale Scientific Data Processing with Data-Parallel Primitives
| Autor: | Zhi-Rong Lin, Ko-Chih Wang, Hao-Yi Yang |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
data-parallel primitive
Numerical Analysis Data processing Industrial engineering. Management engineering Computer science Parallel algorithm distribution-based approach scientific dataset QA75.5-76.95 T55.4-60.8 External Data Representation Mixture model computer.software_genre Theoretical Computer Science large-scale data processing Computational Mathematics Software portability Computational Theory and Mathematics parallel algorithm Electronic computers. Computer science Histogram Use case Data mining Focus (optics) computer |
| Zdroj: | Algorithms Volume 14 Issue 10 Algorithms, Vol 14, Iss 285, p 285 (2021) |
| ISSN: | 1999-4893 |
| DOI: | 10.3390/a14100285 |
| Popis: | The use of distribution-based data representation to handle large-scale scientific datasets is a promising approach. Distribution-based approaches often transform a scientific dataset into many distributions, each of which is calculated from a small number of samples. Most of the proposed parallel algorithms focus on modeling single distributions from many input samples efficiently, but these may not fit the large-scale scientific data processing scenario because they cannot utilize computing resources effectively. Histograms and the Gaussian Mixture Model (GMM) are the most popular distribution representations used to model scientific datasets. Therefore, we propose the use of multi-set histogram and GMM modeling algorithms for the scenario of large-scale scientific data processing. Our algorithms are developed by data-parallel primitives to achieve portability across different hardware architectures. We evaluate the performance of the proposed algorithms in detail and demonstrate use cases for scientific data processing. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|