Conflicting Bundles: Adapting Architectures Towards the Improved Training of Deep Neural Networks
| Autor: | David Peer, Antonio Jose Rodríguez-Sánchez, Sebastian Stabinger |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Artificial neural network Computer science business.industry Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition Inference 02 engineering and technology 010501 environmental sciences 01 natural sciences Machine Learning (cs.LG) Task (project management) Identification (information) Memory management Empirical research Metric (mathematics) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Artificial intelligence Layer (object-oriented design) business 0105 earth and related environmental sciences |
| Zdroj: | WACV |
| DOI: | 10.1109/wacv48630.2021.00030 |
| Popis: | Designing neural network architectures is a challenging task and knowing which specific layers of a model must be adapted to improve the performance is almost a mystery. In this paper, we introduce a novel theory and metric to identify layers that decrease the test accuracy of the trained models, this identification is done as early as at the beginning of training. In the worst-case, such a layer could lead to a network that can not be trained at all. More precisely, we identified those layers that worsen the performance because they produce conflicting training bundles as we show in our novel theoretical analysis, complemented by our extensive empirical studies. Based on these findings, a novel algorithm is introduced to remove performance decreasing layers automatically. Architectures found by this algorithm achieve a competitive accuracy when compared against the state-of-the-art architectures. While keeping such high accuracy, our approach drastically reduces memory consumption and inference time for different computer vision tasks. Comment: Accepted at WACV2021 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|