Bifrost: a Python/C++ Framework for High-Throughput Stream Processing in Astronomy
| Autor: | Miles Cranmer, Barsdell, Benjamin R., Price, Danny C., Dowell, Jayce, Garsden, Hugh, Dike, Veronica, Eftekhari, Tarraneh, Hegedus, Alexander M., Malins, Joseph, Obenberger, Kenneth S., Schinzel, Frank, Stovall, Kevin, Taylor, Gregory B., Greenhill, Lincoln J. |
|---|---|
| Předmět: |
FOS: Computer and information sciences
Physics - Instrumentation and Detectors Computer Science - Distributed Parallel and Cluster Computing FOS: Physical sciences Distributed Parallel and Cluster Computing (cs.DC) Instrumentation and Detectors (physics.ins-det) Astrophysics - Instrumentation and Methods for Astrophysics Instrumentation and Methods for Astrophysics (astro-ph.IM) |
| Zdroj: | NASA Astrophysics Data System |
| Popis: | Radio astronomy observatories with high throughput back end instruments require real-time data processing. While computing hardware continues to advance rapidly, development of real-time processing pipelines remains difficult and time-consuming, which can limit scientific productivity. Motivated by this, we have developed Bifrost: an open-source software framework for rapid pipeline development. Bifrost combines a high-level Python interface with highly efficient reconfigurable data transport and a library of computing blocks for CPU and GPU processing. The framework is generalizable, but initially it emphasizes the needs of high-throughput radio astronomy pipelines, such as the ability to process data buffers as if they were continuous streams, the capacity to partition processing into distinct data sequences (e.g., separate observations), and the ability to extract specific intervals from buffered data. Computing blocks in the library are designed for applications such as interferometry, pulsar dedispersion and timing, and transient search pipelines. We describe the design and implementation of the Bifrost framework and demonstrate its use as the backbone in the correlation and beamforming back end of the Long Wavelength Array station in the Sevilleta National Wildlife Refuge, NM. 25 pages, 13 figures, submitted to JAI. For the code, see https://github.com/ledatelescope/bifrost |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|