Towards High-End Scalability on Biologically-Inspired Computational Models
| Autor: | Alejandro Javier Wainselboim, Bonifacio Silvano Zanutto, George K. Thiruvathukal, Silvio Rizzi, Dario Dematties |
|---|---|
| Přispěvatelé: | Foster, Ian, Joubert, Gerhard R., Kučera, Luděk, Nagel, Wolfgang E., Peters, Frans |
| Rok vydání: | 2020 |
| Předmět: | |
| Zdroj: | PARCO CONICET Digital (CONICET) Consejo Nacional de Investigaciones Científicas y Técnicas instacron:CONICET |
| DOI: | 10.3233/apc200077 |
| Popis: | The interdisciplinary field of neuroscience has made significant progress in recent decades, providing the scientific community in general with a new level of understanding on how the brain works beyond the store-and-fire model found in traditional neural networks. Meanwhile, Machine Learning (ML) based on established models has seen a surge of interest in the High Performance Computing (HPC) community, especially through the use of high-end accelerators, such as Graphical Processing Units(GPUs), including HPC clusters of same. In our work, we are motivated to exploit these high-performance computing developments and understand the scaling challenges for new–biologically inspired–learning models on leadership-class HPC resources. These emerging models feature sparse and random connectivity profiles that map to more loosely-coupled parallel architectures with a large number of CPU cores per node. Contrasted with traditional ML codes, these methods exploit loosely-coupled sparse data structures as opposed to tightly-coupled dense matrix computations, which benefit from SIMD-style parallelism found on GPUs. In this paper we introduce a hybrid Message Passing Interface (MPI) and Open Multi-Processing (OpenMP) parallelization scheme to accelerate and scale our computational model based on the dynamics of cortical tissue. We ran computational tests on a leadership class visualization and analysis cluster at Argonne National Laboratory. We include a study of strong and weak scaling, where we obtained parallel efficiency measures with a minimum above 87% and a maximum above 97% for simulations of our biologically inspired neural network on up to 64 computing nodes running 8 threads each. This study shows promise of the MPI+OpenMP hybrid approach to support flexible and biologically-inspired computational experimental scenarios. In addition, we present the viability in the application of these strategies in high-end leadership computers in the future. Fil: Dematties, Dario Jesus. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica; Argentina Fil: Thiruvathukal, George K.. University of Chicago; Estados Unidos Fil: Rizzi, Silvio. Argonne National Laboratory; Estados Unidos Fil: Wainselboim, Alejandro Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mendoza. Instituto de Ciencias Humanas, Sociales y Ambientales; Argentina Fil: Zanutto, Bonifacio Silvano. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|