Operation and Topology Aware Fast Differentiable Architecture Search
| Autor: | Shahid Siddiqui, Theocharis Theocharides, Christos Kyrkou |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
Network architecture
Speedup Computational complexity theory Computer science Word error rate Topology (electrical circuits) 02 engineering and technology 010501 environmental sciences Topology 01 natural sciences neural networks differentiable neural architecture search image classification deep learning Kernel (linear algebra) 0202 electrical engineering electronic engineering information engineering Search cost 020201 artificial intelligence & image processing 0105 earth and related environmental sciences Block (data storage) |
| Zdroj: | ICPR 2020 25th International Conference on Pattern Recognition (ICPR) |
| Popis: | Differentiable architecture search (DARTS) has gained significant attention amongst neural architecture search approaches due to its effectiveness in finding competitive network architectures with affordable computational complexity. However, DARTS’ search space is designed such that even a randomly sampled architecture performs reasonably well. Moreover, due to the complexity of search architectural building block or cell, it is unclear whether these are certain operations or the cell topology that contributes most to achieving higher final accuracy. In this work, we dissect the DARTS’s search space to understand which components are most effective in producing better architectures. Our experiments show that: (1) Good architectures can be discovered regardless of the search network depth; (2) Seperable convolution with 3x3 kernel is the most effective operation in this search space; and (3) The cell topology also has substantial effect on the accuracy. Based on these insights, we propose an efficient search approach referred to as eDARTS, which searches on a pre-specified cell having good topology with increased attention to important operations, using a shallow search supernet. Moreover, we propose some optimizations for eDARTS that significantly speed up the search as well as alleviate the well known skip connection aggregation problem of DARTS. eDARTS achieves an error rate of 2.53% on CIFAR-10 using a 3.1M parameters model whereas the search cost is less than 30 minutes. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|