ParaView + Alya + D8tree: Integrating high performance computing and high performance data analytics
| Autor: | Cesare Cugnasco, Jordi Torres, Eduard Ayguadé, Jesús Labarta, Fernando M. Cucchietti, Yolanda Becerra, Guillaume Houzeaux, Mariano Vázquez, Antoni Artigues |
|---|---|
| Přispěvatelé: | Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors, Barcelona Supercomputing Center, Universitat Politècnica de Catalunya. CAP - Grup de Computació d'Altes Prestacions |
| Jazyk: | angličtina |
| Rok vydání: | 2017 |
| Předmět: |
Speedup
Computer science Distributed computing Visualització de la informació NoSQL computer.software_genre 01 natural sciences Bottleneck 010305 fluids & plasmas Big data Data visualization Information visualization 0103 physical sciences 0101 mathematics Informàtica::Arquitectura de computadors [Àrees temàtiques de la UPC] Key-Value data stores General Environmental Science business.industry Macrodades Supercomputer Visualization 010101 applied mathematics Scalability HPC Data analysis General Earth and Planetary Sciences Data mining High performance computing business computer HPDA Càlcul intensiu (Informàtica) |
| Zdroj: | Recercat. Dipósit de la Recerca de Catalunya instname UPCommons. Portal del coneixement obert de la UPC Universitat Politècnica de Catalunya (UPC) ICCS |
| ISSN: | 2011-0006 |
| Popis: | Large scale time-dependent particle simulations can generate massive amounts of data, making it so that storing the results is often the slowest phase and the primary time bottleneck of the simulation. Furthermore, analysing this amount of data with traditional tools has become increasingly challenging, and it is often virtually impossible to have a visual representation of the full set. We propose a novel architecture that integrates an HPC-based multi-physics simulation code, a NoSQL database, and a data analysis and visualisation application. The goals are two: On the one hand, we aim to speed up the simulations taking advantage of the scalability of key-value data stores, while at the same time enabling real-time approximated data visualisation and interactive exploration. On the other hand, we want to make it efficient to explore and analyse the large data base of results produced. Therefore, this work represents a clear example of integrating High Performance Computing with High Performance Data Analytics. Our prototype proves the validity of our approach and shows great performance improvements. Indeed, we reduced by 67.5% the time to store the simulation while we made real-time queries run 52 times faster than alternative solutions. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 720270 (HBP SGA1). It is also partially supported by the grant SEV-2011-00067 of Severo Ochoa Program, the TIN2015-65316-P project, with funding from the Spanish Ministry of Economy and Competitivity, the European Union FEDER funds, and the SGR 2014-SGR-1051. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|