TaxoNN: A Light-Weight Accelerator for Deep Neural Network Training
| Autor: | Kossar Pourahmadi, Parsa Nooralinejad, Ahmad Khonsari, M. Hassan Najafi, Dara Rahmati, Reza Hojabr, Kamyar Givaki |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Computer science Heuristic (computer science) Feature extraction Inference 02 engineering and technology Machine learning computer.software_genre 01 natural sciences Machine Learning (cs.LG) Hardware Architecture (cs.AR) 0103 physical sciences 0202 electrical engineering electronic engineering information engineering Neural and Evolutionary Computing (cs.NE) Computer Science - Hardware Architecture 010302 applied physics Training set Artificial neural network business.industry Training (meteorology) Process (computing) Computer Science - Neural and Evolutionary Computing 020202 computer hardware & architecture Stochastic gradient descent Artificial intelligence business computer |
| Zdroj: | ISCAS |
| DOI: | 10.1109/iscas45731.2020.9181001 |
| Popis: | Emerging intelligent embedded devices rely on Deep Neural Networks (DNNs) to be able to interact with the real-world environment. This interaction comes with the ability to retrain DNNs, since environmental conditions change continuously in time. Stochastic Gradient Descent (SGD) is a widely used algorithm to train DNNs by optimizing the parameters over the training data iteratively. In this work, first we present a novel approach to add the training ability to a baseline DNN accelerator (inference only) by splitting the SGD algorithm into simple computational elements. Then, based on this heuristic approach we propose TaxoNN, a light-weight accelerator for DNN training. TaxoNN can easily tune the DNN weights by reusing the hardware resources used in the inference process using a time-multiplexing approach and low-bitwidth units. Our experimental results show that TaxoNN delivers, on average, 0.97% higher misclassification rate compared to a full-precision implementation. Moreover, TaxoNN provides 2.1$\times$ power saving and 1.65$\times$ area reduction over the state-of-the-art DNN training accelerator. Comment: Accepted to ISCAS 2020. 5 pages, 5 figures |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|