HeTraX: Energy Efficient 3D Heterogeneous Manycore Architecture for Transformer Acceleration

Autor:	Dhingra, Pratyush, Doppa, Janardhan Rao, Pande, Partha Pratim
Rok vydání:	2024
Předmět:	Computer Science - Hardware Architecture Computer Science - Machine Learning B.0
Druh dokumentu:	Working Paper
Popis:	Transformers have revolutionized deep learning and generative modeling to enable unprecedented advancements in natural language processing tasks and beyond. However, designing hardware accelerators for executing transformer models is challenging due to the wide variety of computing kernels involved in the transformer architecture. Existing accelerators are either inadequate to accelerate end-to-end transformer models or suffer notable thermal limitations. In this paper, we propose the design of a three-dimensional heterogeneous architecture referred to as HeTraX specifically optimized to accelerate end-to-end transformer models. HeTraX employs hardware resources aligned with the computational kernels of transformers and optimizes both performance and energy. Experimental results show that HeTraX outperforms existing state-of-the-art by up to 5.6x in speedup and improves EDP by 14.5x while ensuring thermally feasibility. Comment: Presented at ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED-24)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2408.03397 Zobrazit plný text záznamu View this record from Arxiv