Energy Efficient Neural Computing: A Study of Cross-Layer Approximations
| Autor: | Gopalakrishnan Srinivasan, Akhilesh Jaiswal, Priyadarshini Panda, Kaushik Roy, Bing Han, Syed Shakib Sarwar, Parami Wijesinghe, Anand Raghunathan |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Artificial neural network
Computer science Computation Approximation algorithm 020206 networking & telecommunications 02 engineering and technology 020202 computer hardware & architecture Computer engineering 0202 electrical engineering electronic engineering information engineering Multiplication Pruning (decision trees) Electrical and Electronic Engineering Energy (signal processing) MNIST database Efficient energy use |
| Zdroj: | IEEE Journal on Emerging and Selected Topics in Circuits and Systems. 8:796-809 |
| ISSN: | 2156-3365 2156-3357 |
| DOI: | 10.1109/jetcas.2018.2835809 |
| Popis: | Deep neural networks (DNNs) have emerged as the state-of-the-art technique in a wide range of machine learning tasks for analytics and computer vision in the next generation of embedded (mobile, IoT, and wearable) devices. Despite their success, they suffer from high energy requirements. In recent years, the inherent error resiliency of DNNs has been exploited by introducing approximations at either the algorithmic or the hardware levels (individually) to obtain energy savings while incurring tolerable accuracy degradation. However, there is a need for investigating the overall energy-accuracy trade-offs arising from the introduction of approximations at different levels in complex DNNs. We perform a comprehensive analysis to determine the effectiveness of cross-layer approximations for the energy-efficient realization of large-scale DNNs. The approximations considered are as follows: 1) use of lower complexity networks (containing lesser number of layers and/or neurons per layer); 2) pruning of synaptic weights; 3) approximate multiplication operation in the neuronal multiply-and-accumulate computation; and 4) approximate write/read operations to/from the synaptic memory. Our experiments on recognition benchmarks (MNIST and CIFAR10) show that cross-layer approximation provides substantial improvements in energy efficiency for different accuracy/quality requirements. Furthermore, we propose a synergistic framework for combining the approximation techniques to achieve maximal energy benefits from approximate DNNs. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|