Zobrazeno 1 - 4
of 4
pro vyhledávání: '"Abdukhakimov, Farshed"'
This paper investigates the global convergence of stepsized Newton methods for convex functions. We propose several simple stepsize schedules with fast global convergence guarantees, up to $\mathcal{O} (k^{-3})$, nearly matching lower complexity boun
Externí odkaz:
http://arxiv.org/abs/2405.18926
Adaptive optimization methods are widely recognized as among the most popular approaches for training Deep Neural Networks (DNNs). Techniques such as Adam, AdaGrad, and AdaHessian utilize a preconditioner that modifies the search direction by incorpo
Externí odkaz:
http://arxiv.org/abs/2312.17369
Stochastic Gradient Descent (SGD) is one of the many iterative optimization methods that are widely used in solving machine learning problems. These methods display valuable properties and attract researchers and industrial machine learning engineers
Externí odkaz:
http://arxiv.org/abs/2310.02093
Autor:
Gasnikov, Alexander, Novitskii, Anton, Novitskii, Vasilii, Abdukhakimov, Farshed, Kamzolov, Dmitry, Beznosikov, Aleksandr, Takáč, Martin, Dvurechensky, Pavel, Gu, Bin
Gradient-free/zeroth-order methods for black-box convex optimization have been extensively studied in the last decade with the main focus on oracle calls complexity. In this paper, besides the oracle complexity, we focus also on iteration complexity,
Externí odkaz:
http://arxiv.org/abs/2201.12289