| Autor: |
Scot H. Rider, Martin Lutz, Moriyoshi Ohara, Pong-Fei Lu, Monodeep Kar, Xiao Sun, Kailash Gopalakrishnan, Jie Yang, Hoang Tran, Wei Wang, Michael A. Guillorn, Marcel Schaal, Ankur Agrawal, Xin Zhang, Joel Abraham Silberman, Sunil Shukla, Nianzheng Cao, James Bonano, Zhibin Ren, Sanchari Sen, Siyu Koswatta, Kyu-hyoun Kim, Mingu Kang, Swagath Venkataramani, Eri Ogawa, Vijayalakshmi Srinivasan, Hiroshi Inoue, Matt Ziegler, Howard M. Haynie, Shubham Jain, Vinay Velji Shah, Allison Allain, Jintao Zhang, Matthew Cohen, Jungwook Choi, Kerstin Schelm, Jinwook Oh, Li Yulong, Chia-Yu Chen, Ching Zhou, Naigang Wang, Jinwook Jung, Sae Kyu Lee, Silvia Melitta Mueller, Kazuaki Ishizaki, Bruce M. Fleischer, Michael R. Scheuermann, Vidhi Zalani, Brian W. Curran, Leland Chang, Mauricio J. Serrano, Ashish Ranjan, Alberto Mannari, Robert Casatuta |
| Rok vydání: |
2021 |
| Předmět: |
|
| Zdroj: |
ISCA |
| Popis: |
The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2 RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications. |
| Databáze: |
OpenAIRE |
| Externí odkaz: |
|
