Hardware Implementation of a Softmax-Like Function for Deep Learning

Autor: Ioannis Kouretas, Vassilis Paliouras
Jazyk: angličtina
Rok vydání: 2020
Předmět:
Zdroj: Technologies, Vol 8, Iss 3, p 46 (2020)
Druh dokumentu: article
ISSN: 2227-7080
DOI: 10.3390/technologies8030046
Popis: In this paper a simplified hardware implementation of a CNN softmax-like layer is proposed. Initially the softmax activation function is analyzed in terms of required numerical accuracy and certain optimizations are proposed. A proposed adaptable hardware architecture is evaluated in terms of the introduced error due to the proposed softmax-like function. The proposed architecture can be adopted to the accuracy required by the application by retaining or eliminating certain terms of the approximation thus allowing to explore accuracy for complexity trade-offs. Furthermore, the proposed circuits are synthesized in a 90 nm 1.0 V CMOS standard-cell library using Synopsys Design Compiler. Comparisons reveal that significant reduction is achieved in area × delay and power × delay products for certain cases, respectively, over prior art. Area and power savings are achieved with respect to performance and accuracy.
Databáze: Directory of Open Access Journals