Versatile kernel reactivation for deep convolutional neural networks

Autor:	Jeong Jun Lee, Hyun Kim
Jazyk:	angličtina
Rok vydání:	2022
Předmět:	Optical image and video signal processing Computer vision and image processing techniques Neural nets Electrical engineering. Electronics. Nuclear engineering TK1-9971
Zdroj:	Electronics Letters, Vol 58, Iss 19, Pp 723-725 (2022)
Druh dokumentu:	article
ISSN:	1350-911X 0013-5194
DOI:	10.1049/ell2.12578
Popis:	Abstract Several networks exhibiting excellent performance in the field of computer vision still suffer from the challenge of over‐parameterization. Over‐parameterized networks have several parameters that cannot be trained, owing to the poor backpropagation process caused by a small L1 norm, and these parameters suppress the potential for network performance improvement. This study proposes a kernel reactivation method to improve network performance by reusing invalid kernels that cannot be utilized for training because of the small L1 norm. The results indicate that the accuracy of Cifar‐10 in ResNet‐110 is improved by 0.94% compared to the baseline, and the top‐1 and top‐5 accuracies of Tiny‐ImageNet in ResNet‐50 are improved by 1.87% and 1.03% compared to the baseline, respectively.
Databáze:	Directory of Open Access Journals
Externí odkaz:	https://doaj.org/article/5e0d9abdb3314c1298a858880a15c3b6 Zobrazit plný text záznamu Plný text View record in DOAJ