Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Autor:	Huang, Hanxun, Wang, Yisen, Erfani, Sarah Monazam, Gu, Quanquan, Bailey, James, Ma, Xingjun
Rok vydání:	2021
Předmět:	Computer Science - Machine Learning Computer Science - Computer Vision and Pattern Recognition Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. A range of defense methods have been proposed to train adversarially robust DNNs, among which adversarial training has demonstrated promising results. However, despite preliminary understandings developed for adversarial training, it is still not clear, from the architectural perspective, what configurations can lead to more robust DNNs. In this paper, we address this gap via a comprehensive investigation on the impact of network width and depth on the robustness of adversarially trained DNNs. Specifically, we make the following key observations: 1) more parameters (higher model capacity) does not necessarily help adversarial robustness; 2) reducing capacity at the last stage (the last group of blocks) of the network can actually improve adversarial robustness; and 3) under the same parameter budget, there exists an optimal architectural configuration for adversarial robustness. We also provide a theoretical analysis explaning why such network configuration can help robustness. These architectural insights can help design adversarially robust DNNs. Code is available at \url{https://github.com/HanxunH/RobustWRN}. Comment: NeurIPS 2021
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2110.03825 Zobrazit plný text záznamu View this record from Arxiv