Scalable streaming tools for analyzing N-body simulations : finding halos and investigating excursion sets in one pass
| Autor: | Ivkin, Nikita, Liu, Zaoxing, Yang, Lin F., Kumar, Srinivas Suresh, Lemson, Gerard, Neyrinck, Mark, Szalay, Alexander S., Braverman, Vladimir, Budavari, Tamas |
|---|---|
| Rok vydání: | 2018 |
| Předmět: | |
| Zdroj: | Astronomy and computing, 2018, Vol.23, pp.166-179 [Peer Reviewed Journal] |
| Popis: | Cosmological $N$-body simulations play a vital role in studying models for the evolution of the Universe. To compare to observations and make a scientific inference, statistic analysis on large simulation datasets, e.g., finding halos, obtaining multi-point correlation functions, is crucial. However, traditional in-memory methods for these tasks do not scale to the datasets that are forbiddingly large in modern simulations. Our prior paper proposes memory-efficient streaming algorithms that can find the largest halos in a simulation with up to $10^9$ particles on a small server or desktop. However, this approach fails when directly scaling to larger datasets. This paper presents a robust streaming tool that leverages state-of-the-art techniques on GPU boosting, sampling, and parallel I/O, to significantly improve performance and scalability. Our rigorous analysis of the sketch parameters improves the previous results from finding the centers of the $10^3$ largest halos to $\sim 10^4-10^5$, and reveals the trade-offs between memory, running time and number of halos. Our experiments show that our tool can scale to datasets with up to $\sim 10^{12}$ particles while using less than an hour of running time on a single GPU Nvidia GTX 1080. Comment: preprint |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|