Spatial Shift Point-Wise Quantization
| Autor: | Kyong-Ha Lee, Eunhui Kim |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
Pointwise
General Computer Science Compact neural network design Computer science Quantization (signal processing) 05 social sciences General Engineering 010501 environmental sciences lightweight modeling 01 natural sciences Convolutional neural network convolution neural network Spatial shift 0502 economics and business General Materials Science quantization lcsh:Electrical engineering. Electronics. Nuclear engineering 050207 economics Algorithm lcsh:TK1-9971 0105 earth and related environmental sciences |
| Zdroj: | IEEE Access, Vol 8, Pp 207683-207690 (2020) |
| ISSN: | 2169-3536 |
| Popis: | Deep neural networks (DNN) have been applied to numerous artificial-intelligence applications because of their remarkable accuracy. However, computational requirements for deep neural networks are recently skyrocketing far beyond the Moore’s Law. In addition to the importance of accuracy, the industry’s demand for efficiency in model learning process is increasing. This has led to various attempts to make DNNs more lightweight. Hence, we propose a modeling technique that applies lightweight convolutional neural networks (CNN) to handle the model-learning processes for DNNs. The proposed spatial-shift pointwise quantization (SSPQ) model elegantly combines compact network-design techniques to revitalize DNN quantization efficiency with little accuracy loss. We set the depths of our SSPQ model to 20, 34, and 50 to test against CIFAR10, CIFAR100, and ImageNet datasets, respectively. By applying SSPQ20 to the CIFAR10 dataset, we reduced accuracy degradation by 2.95%, while reducing the number of parameters $8.7\times $ . For the same dataset, our “wide” SSPQ20 variant reduced training parameters $1.96\times $ , compared with the ResNet20 model architecture, which provided a 0.2% improvement. By applying SSPQ34 to the CIFAR100 dataset, we successfully reduced the number of learning parameters $8\times $ , compared with the ResNet34 model with an accuracy degradation of 4.57%. By applying SSPQ50 to the ImageNet dataset, we successfully reduced the number of parameters $10.2\times $ over ResNet50 with an accuracy degradation of only 2.68%. Therefore, using only 9.78MB of learning parameter, the SSPQ50 model guarantees 73.6% accuracy. This is an improved performance by 1.46% in terms of accuracy and 3.62MB in terms of model size compared to the MobileNetV2 lightweight model. The source code is available at https://github.com/Eunhui-Kim/SSPQ . |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|