Progressive Learning of Low-Precision Networks for Image Classification
| Autor: | Huang Xuan, Zhengguang Zhou, Wengang Zhou, Houqiang Li, Xiaoyu Wang, Xutao Lv |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Artificial neural network
Contextual image classification Computer science business.industry Deep learning Topology (electrical circuits) 02 engineering and technology Machine learning computer.software_genre Computer Science Applications Convolution Signal Processing 0202 electrical engineering electronic engineering information engineering Media Technology 020201 artificial intelligence & image processing Artificial intelligence Electrical and Electronic Engineering Layer (object-oriented design) business computer |
| Zdroj: | IEEE Transactions on Multimedia. 23:871-882 |
| ISSN: | 1941-0077 1520-9210 |
| Popis: | Recent years have witnessed a great advance of deep learning in a variety of vision tasks. Many state-of-the-art deep neural networks suffer from large size and high complexity, which makes them difficult to deploy in resource-limited platforms such as mobile devices. To this end, low-precision neural networks are widely studied that quantize weights or activations into the low-bit format. Although efficient, low-precision networks are usually difficult to train and encounter severe accuracy degradation. In this paper, we propose a new training strategy based on progressive learning for image classification. First, we equip each low-precision convolutional layer with an ancillary full-precision convolutional layer based on a low-precision network structure. Second, a decay method is introduced to reduce the output of the added full-precision convolution gradually, which keeps the resulting topology structure the same as the original low-precision convolution. Extensive experiments on SVHN, CIFAR and ILSVRC-2012 datasets reveal that the proposed method can bring faster convergence and higher accuracy for low-precision neural networks. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|