Autor:	Akrout, Mohamed, Tweed, Douglas
Rok vydání:	2021
Předmět:	Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Why does the Adam optimizer work so well in deep-learning applications? Adam's originators, Kingma and Ba, presented a mathematical argument that was meant to help explain its success, but Bock and colleagues have since reported that a key piece is missing from that argument $-$ an unproven lemma which we will call Bock's conjecture. Here we show that this conjecture is false, but we prove a modified version of it $-$ a generalization of a result of Reddi and colleagues $-$ which can take its place in analyses of Adam. Comment: Accepted to the Optimization Algorithms and Machine Learning (ICOAML) conference
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2111.08162 Zobrazit plný text záznamu View this record from Arxiv