Zobrazeno 1 - 4
of 4
pro vyhledávání: '"KOROKO, ABDOULAYE"'
Autor:
Koroko, Abdoulaye, Anciaux-Sedrakian, Ani, Gharbia, Ibtihel Ben, Garès, Valérie, Haddou, Mounir, Tran, Quang Huy
As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing and inverting the Fisher Information Matrix (FIM), eff
Externí odkaz:
http://arxiv.org/abs/2303.18083
Autor:
Koroko, Abdoulaye, Anciaux-Sedrakian, Ani, Gharbia, Ibtihel Ben, Garès, Valérie, Haddou, Mounir, Tran, Quang Huy
Several studies have shown the ability of natural gradient descent to minimize the objective function more efficiently than ordinary gradient descent based methods. However, the bottleneck of this approach for training deep neural networks lies in th
Externí odkaz:
http://arxiv.org/abs/2201.10285
Autor:
Koroko Abdoulaye, Anciaux-Sedrakian Ani, Gharbia Ibtihel Ben, Garès Valérie, Haddou Mounir, Tran Quang Huy
Publikováno v:
ESAIM: Proceedings and Surveys, Vol 73, Pp 218-237 (2023)
We design four novel approximations of the Fisher Information Matrix (FIM) that plays a central role in natural gradient descent methods for neural networks. The newly proposed approximations are aimed at improving Martens and Grosse’s Kronecker-fa
Externí odkaz:
https://doaj.org/article/8a4fc72eae9e421f84c52e8da9df936d
Akademický článek
Tento výsledek nelze pro nepřihlášené uživatele zobrazit.
K zobrazení výsledku je třeba se přihlásit.
K zobrazení výsledku je třeba se přihlásit.