A Hardware-efficient Weight Sampling Circuit for Bayesian Neural Networks
| Autor: | Masato Motomura, Tetsuya Asai, Yuki Hirayama, Shinya Takamaeda |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Artificial neural network
Computational complexity theory business.industry Computer science Deep learning Gaussian 05 social sciences Sampling (statistics) 010501 environmental sciences 01 natural sciences symbols.namesake Function approximation 0502 economics and business Lookup table symbols Probability distribution Artificial intelligence 050207 economics business Computer hardware 0105 earth and related environmental sciences |
| Zdroj: | International Journal of Networking and Computing. 10:84-93 |
| ISSN: | 2185-2847 2185-2839 |
| DOI: | 10.15803/ijnc.10.2_84 |
| Popis: | The main problems of deep learning are requiring a large amount of data for learning, and prediction with excessive confidence. A Bayesian neural network (BNN), in which a Bayesian approach is incorporated into a neural network (NN), has drawn attention as a method for solving these problems. In a BNN, the probability distribution is assumed for the weight, in contrast to a conventional NN, in which the weight is point estimated. This makes it possible to obtain the prediction as a distribution and to evaluate how uncertain the prediction is. However, a BNN has more computational complexity and a greater number of parameters than an NN. To obtain an inference result as a distribution, a BNN uses weight sampling to generate the respective weight values, and thus, a BNN accelerator requires weight sampling hardware based on a random number generator in addition to the standard components of a deep learning neural network accelerator. Therefore, the throughput of weight sampling must be sufficiently high at a low hardware resource cost. We propose a resource-efficient weight sampling method using inversion transform sampling and a lookup-table (LUT)-based function approximation for hardware implementation of a BNN. Inversion transform sampling simplifies the mechanism of generating a Gaussian random number from a uniform random number provided by a common random number generator, such as a linear feedback shift register. Employing an LUT-based low-bit precision function approximation enables inversion transform sampling to be implemented at a low hardware cost. The evaluation results indicate that this approach effectively reduces the occupied hardware resources while maintaining accuracy and prediction variance equivalent to that with a non-approximated sampling method. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|