BRP-NAS: Prediction-based NAS using GCNs
Autor: | Dudziak, ��ukasz, Chau, Thomas, Abdelfattah, Mohamed S., Lee, Royson, Kim, Hyeji, Lane, Nicholas D. |
---|---|
Jazyk: | angličtina |
Rok vydání: | 2020 |
Předmět: |
Signal Processing (eess.SP)
FOS: Computer and information sciences Computer Science - Machine Learning Statistics - Machine Learning FOS: Electrical engineering electronic engineering information engineering Machine Learning (stat.ML) Electrical Engineering and Systems Science - Signal Processing Machine Learning (cs.LG) |
Popis: | Neural architecture search (NAS) enables researchers to automatically explore broad design spaces in order to improve efficiency of neural networks. This efficiency is especially important in the case of on-device deployment, where improvements in accuracy should be balanced out with computational demands of a model. In practice, performance metrics of model are computationally expensive to obtain. Previous work uses a proxy (e.g., number of operations) or a layer-wise measurement of neural network layers to estimate end-to-end hardware performance but the imprecise prediction diminishes the quality of NAS. To address this problem, we propose BRP-NAS, an efficient hardware-aware NAS enabled by an accurate performance predictor-based on graph convolutional network (GCN). What is more, we investigate prediction quality on different metrics and show that sample efficiency of the predictor-based NAS can be improved by considering binary relations of models and an iterative data selection strategy. We show that our proposed method outperforms all prior methods on NAS-Bench-101 and NAS-Bench-201, and that our predictor can consistently learn to extract useful features from the DARTS search space, improving upon the second-order baseline. Finally, to raise awareness of the fact that accurate latency estimation is not a trivial task, we release LatBench -- a latency dataset of NAS-Bench-201 models running on a broad range of devices. Published at NeurIPS 2020 |
Databáze: | OpenAIRE |
Externí odkaz: |