Shallow Univariate ReLu Networks as Splines: Initialization, Loss Surface, Hessian, & Gradient Flow Dynamics

Autor: Sahs, Justin, Pyle, Ryan, Damaraju, Aneel, Caro, Josue Ortega, Tavaslioglu, Onur, Lu, Andy, Patel, Ankit

Understanding the learning dynamics and inductive bias of neural networks (NNs) is hindered by the opacity of the relationship between NN parameters and the function represented. We propose reparametrizing ReLU NNs as continuous piecewise linear spli

Externí odkaz: http://arxiv.org/abs/2008.01772

Tento výsledek nelze pro nepřihlášené uživatele zobrazit.
K zobrazení výsledku je třeba se přihlásit.

Tento výsledek nelze pro nepřihlášené uživatele zobrazit.
K zobrazení výsledku je třeba se přihlásit.

Shallow Univariate ReLU Networks as Splines: Initialization, Loss Surface, Hessian, and Gradient Flow Dynamics.

Autor: Sahs J; Department of Neuroscience, Baylor College of Medicine, Houston, TX, United States., Pyle R; Department of Neuroscience, Baylor College of Medicine, Houston, TX, United States., Damaraju A; Department of Electrical Engineering, Rice University, Houston, TX, United States., Caro JO; Department of Neuroscience, Baylor College of Medicine, Houston, TX, United States., Tavaslioglu O; Department of Computational and Applied Mathematics, Rice University, Houston, TX, United States., Lu A; Department of Electrical Engineering, Rice University, Houston, TX, United States., Anselmi F; Department of Neuroscience, Baylor College of Medicine, Houston, TX, United States., Patel AB; Department of Neuroscience, Baylor College of Medicine, Houston, TX, United States.; Department of Electrical Engineering, Rice University, Houston, TX, United States.

Publikováno v: Frontiers in artificial intelligence [Front Artif Intell] 2022 May 11; Vol. 5, pp. 889981. Date of Electronic Publication: 2022 May 11 (Print Publication: 2022).