Local Search is a Remarkably Strong Baseline for Neural Architecture Search
Autor: | den Ottelander, Tom, Dushatskiy, Arkadiy, Virgolin, Marco, Bosman, Peter A. N., Ishibuchi, Hisao, Zhang, Qingfu, Cheng, Ran, Li, Ke, Li, Hui, Wang, Handing, Zhou, Aimin |
---|---|
Přispěvatelé: | Centrum Wiskunde & Informatica, Amsterdam (CWI), The Netherlands, Radiotherapy |
Jazyk: | angličtina |
Rok vydání: | 2021 |
Předmět: |
Network complexity
Computer science Evolutionary algorithm 02 engineering and technology Machine learning computer.software_genre 01 natural sciences Random search Multi-objective NAS Search algorithm 0103 physical sciences 0202 electrical engineering electronic engineering information engineering Local search (optimization) 010302 applied physics Contextual image classification business.industry Neural Architecture Search Automation Local Search Benchmark (computing) 020201 artificial intelligence & image processing Artificial intelligence business computer NAS baseline |
Zdroj: | Evolutionary Multi-Criterion Optimization-11th International Conference, EMO 2021, Proceedings, 12654 LNCS, 465-479 Lecture Notes in Computer Science ISBN: 9783030720612 EMO |
ISSN: | 0302-9743 |
Popis: | Neural Architecture Search (NAS), i.e., the automation of neural network design, has gained much popularity in recent years with increasingly complex search algorithms being proposed. Yet, solid comparisons with simple baselines are often missing. At the same time, recent retrospective studies have found many new algorithms to be no better than random search (RS). In this work we consider the use of a simple Local Search (LS) algorithm for NAS. We particularly consider a multi-objective NAS formulation, with network accuracy and network complexity as two objectives, as understanding the trade-off between these two objectives is arguably among the most interesting aspects of NAS. The proposed LS algorithm is compared with RS and two evolutionary algorithms (EAs), as these are often heralded as being ideal for multi-objective optimization. To promote reproducibility, we create and release two benchmark datasets, named MacroNAS-C10 and -C100, containing 200K saved network evaluations for two established image classification tasks, CIFAR-10 and CIFAR-100. Our benchmarks are designed to be complementary to existing benchmarks, especially in that they are better suited for multi-objective search. We additionally consider a version of the problem with a much larger architecture space. While we find and show that the considered algorithms explore the search space in fundamentally different ways, we also find that LS substantially outperforms RS and even performs nearly as good as state-of-the-art EAs. We believe that this provides strong evidence that LS is truly a competitive baseline for NAS against which new NAS algorithms should be benchmarked. |
Databáze: | OpenAIRE |
Externí odkaz: |