Memory access patterns
| Autor: | Amnon Barak, Tal Ben-Nun, Ely Levy, Eri Rubin |
|---|---|
| Rok vydání: | 2015 |
| Předmět: |
Computer science
business.industry Deep learning Distributed computing Uniform memory access Parallel computing Load balancing (computing) Program optimization Memory map Computer Science::Performance Instruction set Memory management CUDA Pinned memory Programming paradigm Artificial intelligence business |
| Zdroj: | SC |
| DOI: | 10.1145/2807591.2807611 |
| Popis: | With the increased popularity of multi-GPU nodes in modern HPC clusters, it is imperative to develop matching programming paradigms for their efficient utilization. In order to take advantage of the local GPUs and the low-latency high-throughput interconnects that link them, programmers need to meticulously adapt parallel applications with respect to load balancing, boundary conditions and device synchronization. This paper presents MAPS-Multi, an automatic multi-GPU partitioning framework that distributes the workload based on the underlying memory access patterns. The framework consists of host- and device-level APIs that allow programs to efficiently run on a variety of GPU and multi-GPU architectures. The framework implements several layers of code optimization, device abstraction, and automatic inference of inter-GPU memory exchanges. The paper demonstrates that the performance of MAPS-Multi achieves near-linear scaling on fundamental computational operations, as well as real-world applications in deep learning and multivariate analysis. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|