Mini-apps for high performance data analysis
| Autor: | Michael A. Matheson, Ramakrishnan Kannan, Sreenivas R. Sukumar, Seung-Hwan Lim |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
business.industry
Computer science Big data 0102 computer and information sciences 02 engineering and technology computer.software_genre 01 natural sciences Workflow Computer architecture 010201 computation theory & mathematics 020204 information systems Scalability 0202 electrical engineering electronic engineering information engineering Programming paradigm Operating system Algorithm design Software architecture business computer |
| Zdroj: | IEEE BigData |
| DOI: | 10.1109/bigdata.2016.7840756 |
| Popis: | Scaling-up scientific data analysis and machine learning algorithms for data-driven discovery is a grand challenge that we face today. Despite the growing need for analysis from science domains that are generating ‘Big Data’ from instruments and simulations, building high-performance analytical workflows of data-intensive algorithms have been daunting because: (i) the ‘Big Data’ hardware and software architecture landscape is constantly evolving, (ii) newer architectures impose new programming models, and (iii) data-parallel kernels of analysis algorithms and their performance facets on different architectures are poorly understood. To address these problems, we have: (i) identified scalable data-parallel kernels of popular data analysis algorithms, (ii) implemented ‘Mini-Apps’ of those kernels using different programming models (e.g. Map Reduce, MPI, etc.), (iii) benchmarked and validated the performance of the kernels in diverse architectures. In this paper, we discuss two of those Mini-Apps and show the execution of principal component analysis built as a workflow of the Mini-Apps. We show that Mini-Apps enable scientists to (i) write domain-specific data analysis code that scales on most HPC hardware and (ii) and offers the ability (most times with over a 10x speed-up) to analyze data sizes 100 times the size of what off-the-shelf desktop/workstations of today can handle. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|