Platform for Parallel Processing of Intense Experimental Data Flow on Remote Supercomputers
| Autor: | Alexey Masich, Vladislav Shchapov, Grigoriy Masich |
|---|---|
| Rok vydání: | 2015 |
| Předmět: |
Middleware
business.industry Computer science Distributed computing Experimental data Supercomputer Parallel data processing Long fat network Software Parallel processing (DSP implementation) Computer data storage Distributed system General Earth and Planetary Sciences business General Environmental Science Data transmission |
| Zdroj: | Procedia Computer Science. 66:515-524 |
| ISSN: | 1877-0509 |
| Popis: | Modern experimental facilities generate large amounts of data that should be processed, saved to hard disk and presented to the user as fast as possible. However, in-situ data analysis requires technical resources which are often not available. The existence of accessible high-speed networks allows to forward data processing and storage to a remote supercomputer centers and datacenters. These capabilities can be realized through the development of architectural solutions for effective data transmission trough a long-distance high-speed networks, data input/output and data distribution over computers and data storage systems. In this paper, we describe the results of investigations into the development of a software platform for parallel processing of intense experimental data-streams on ICMM UB RAS (Perm) and IMM UB RAS (Yekaterinburg) supercomputers, interconnected by a high-speed network. The reported studies was partially supported by RFBR, research project No. 14-07-96001-r_ural_a and by Program of UD RAS, project No 15-7-1-25. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|