Adaptive Stochastic Optimisation of Nonconvex Composite Objectives

Autor:	Shao, Weijia, Sivrikaya, Fikret, Albayrak, Sahin
Rok vydání:	2022
Předmět:	Mathematics - Optimization and Control Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	In this paper, we propose and analyse a family of generalised stochastic composite mirror descent algorithms. With adaptive step sizes, the proposed algorithms converge without requiring prior knowledge of the problem. Combined with an entropy-like update-generating function, these algorithms perform gradient descent in the space equipped with the maximum norm, which allows us to exploit the low-dimensional structure of the decision sets for high-dimensional problems. Together with a sampling method based on the Rademacher distribution and variance reduction techniques, the proposed algorithms guarantee a logarithmic complexity dependence on dimensionality for zeroth-order optimisation problems. Comment: arXiv admin note: substantial text overlap with arXiv:2208.04579
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2211.11710 Zobrazit plný text záznamu View this record from Arxiv