Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions
| Autor: | Alvin Wan, Peter H. Jin, Sicheng Zhao, Joseph E. Gonzalez, Kurt Keutzer, Noah Golmant, Xiangyu Yue, Bichen Wu, Amir Gholaminejad |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
FOS: Computer and information sciences
Hyperparameter Quadratic growth Artificial neural network Contextual image classification business.industry Computer science Computer Vision and Pattern Recognition (cs.CV) Computation Computer Science - Computer Vision and Pattern Recognition 020206 networking & telecommunications 02 engineering and technology Convolution Kernel (linear algebra) Kernel (image processing) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Artificial intelligence business Algorithm |
| Zdroj: | CVPR |
| Popis: | Neural networks rely on convolutions to aggregate spatial information. However, spatial convolutions are expensive in terms of model size and computation, both of which grow quadratically with respect to kernel size. In this paper, we present a parameter-free, FLOP-free "shift" operation as an alternative to spatial convolutions. We fuse shifts and point-wise convolutions to construct end-to-end trainable shift-based modules, with a hyperparameter characterizing the tradeoff between accuracy and efficiency. To demonstrate the operation's efficacy, we replace ResNet's 3x3 convolutions with shift-based modules for improved CIFAR10 and CIFAR100 accuracy using 60% fewer parameters; we additionally demonstrate the operation's resilience to parameter reduction on ImageNet, outperforming ResNet family members. We finally show the shift operation's applicability across domains, achieving strong performance with fewer parameters on classification, face verification and style transfer. Comment: Source code will be released afterwards |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|