Band-limited Training and Inference for Convolutional Neural Networks
| Autor: | Adam Dziedzic, John Paparrizos, Sanjay Krishnan, Aaron Elmore, Franklin, Michael J. |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Statistics - Machine Learning Computer Vision and Pattern Recognition (cs.CV) Computer Science::Neural and Evolutionary Computation Computer Science - Computer Vision and Pattern Recognition Machine Learning (stat.ML) Machine Learning (cs.LG) |
| Zdroj: | Aaron Elmore |
| Popis: | The convolutional layers are core building blocks of neural network architectures. In general, a convolutional filter applies to the entire frequency spectrum of the input data. We explore artificially constraining the frequency spectra of these filters and data, called band-limiting, during training. The frequency domain constraints apply to both the feed-forward and back-propagation steps. Experimentally, we observe that Convolutional Neural Networks (CNNs) are resilient to this compression scheme and results suggest that CNNs learn to leverage lower-frequency components. In particular, we found: (1) band-limited training can effectively control the resource usage (GPU and memory); (2) models trained with band-limited layers retain high prediction accuracy; and (3) requires no modification to existing training algorithms or neural network architectures to use unlike other compression schemes. Published at International Conference on Machine Learning (ICML) |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|