Task-based acceleration of bidirectional recurrent neural networks on multi-core architectures
| Autor: | Robin Kumar Sharma, Marc Casas |
|---|---|
| Přispěvatelé: | Universitat Politècnica de Catalunya. Doctorat en Arquitectura de Computadors, Barcelona Supercomputing Center |
| Jazyk: | angličtina |
| Rok vydání: | 2022 |
| Předmět: |
Neural networks (Computer science)
Parallel processing (Electronic computers) Bidirectional recurrent neural networks (BRNNs) Long-short term memory (LSTM) Gated recurrent units (GRU) Processament en paral·lel (Ordinadors) Task parallelism Xarxes neuronals (Informàtica) Deep learning Deep neural network (DNN) Informàtica::Arquitectura de computadors::Arquitectures paral·leles [Àrees temàtiques de la UPC] Aprenentatge profund |
| Zdroj: | UPCommons. Portal del coneixement obert de la UPC Universitat Politècnica de Catalunya (UPC) |
| Popis: | This paper proposes a novel parallel execution model for Bidirectional Recurrent Neural Networks (BRNNs), B-Par (Bidirectional-Parallelization), which exploits data and control dependencies for forward and reverse input computations. B-Par divides BRNN workloads across different parallel tasks by defining input and output dependencies for each RNN cell in both forward and reverse orders. B-Par does not require per-layer barriers to synchronize the parallel execution of BRNNs. We evaluate B-Par considering the TIDIGITS speech database and the Wikipedia data-set. Our experiments indicate that B-Par outperforms the state-of-the-art deep learning frameworks TensorFlow-Keras and Pytorch by achieving up to 2.34× and 9.16× speed-ups, respectively, on modern multi-core CPU architectures while preserving accuracy. Moreover, we analyze in detail aspects like task granularity, locality, or parallel efficiency to illustrate the benefits of B-Par. This work is partially supported by the Generalitat de Catalunya (contract 2017-SGR-1414) and the Spanish Ministry of Science and Technology through the PID2019- 107255GB project. Marc Casas has been supported by the Spanish Ministry of Economy, Industry and Competitiveness under the Ramon y Cajal fellowship No. RYC-2017-23269. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|