Growth and harvest induce essential dynamics in neural networks

Autor:	Tarzan Legović, Ilona M. Kulikovskikh
Rok vydání:	2021
Předmět:	education.field_of_study Artificial neural network Generalization Computer science Population Activation function Dynamics (mechanics) Stability (learning theory) Function (mathematics) Enhanced Data Rates for GSM Evolution Biological system education
Zdroj:	GECCO Companion
Popis:	Training neural networks with faster gradient methods brings them to the edge of stability, proximity to which improves their generalization capability. However, it is not clear how to stably approach the edge. We propose a new activation function to model inner processes inside neurons with single-species population dynamics. The function induces essential dynamics in neural networks with a growth and harvest rate to improve their generalization capability.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_________::fcd681047dd36059858345572ad715b2 https://doi.org/10.1145/3449726.3459421 Zobrazit plný text záznamu