Neural Architecture Search for Lightweight Non-Local Networks
| Autor: | Xiaochen Lian, Xiaojie Jin, Yuyin Zhou, Song Bai, Jieru Mei, Cihang Xie, Alan L. Yuille, Linjie Yang, Qihang Yu, Yingwei Li |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Artificial neural network business.industry Computer science Computer Vision and Pattern Recognition (cs.CV) Deep learning Computer Science - Computer Vision and Pattern Recognition 02 engineering and technology 010501 environmental sciences 01 natural sciences Transformation (function) Computer engineering Block (programming) Search algorithm 0202 electrical engineering electronic engineering information engineering Code (cryptography) 020201 artificial intelligence & image processing Artificial intelligence business 0105 earth and related environmental sciences |
| Zdroj: | CVPR |
| DOI: | 10.1109/cvpr42600.2020.01031 |
| Popis: | Non-Local (NL) blocks have been widely studied in various vision tasks. However, it has been rarely explored to embed the NL blocks in mobile neural networks, mainly due to the following challenges: 1) NL blocks generally have heavy computation cost which makes it difficult to be applied in applications where computational resources are limited, and 2) it is an open problem to discover an optimal configuration to embed NL blocks into mobile neural networks. We propose AutoNL to overcome the above two obstacles. Firstly, we propose a Lightweight Non-Local (LightNL) block by squeezing the transformation operations and incorporating compact features. With the novel design choices, the proposed LightNL block is 400x computationally cheaper} than its conventional counterpart without sacrificing the performance. Secondly, by relaxing the structure of the LightNL block to be differentiable during training, we propose an efficient neural architecture search algorithm to learn an optimal configuration of LightNL blocks in an end-to-end manner. Notably, using only 32 GPU hours, the searched AutoNL model achieves 77.7% top-1 accuracy on ImageNet under a typical mobile setting (350M FLOPs), significantly outperforming previous mobile models including MobileNetV2 (+5.7%), FBNet (+2.8%) and MnasNet (+2.1%). Code and models are available at https://github.com/LiYingwei/AutoNL. Comment: CVPR 2020. Project page: https://github.com/LiYingwei/AutoNL |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|