The Mini-batch Stochastic Conjugate Algorithms with the unbiasedness and Minimized Variance Reduction

Autor:	Gao, Feifei, Kou, Caixia
Rok vydání:	2023
Předmět:	Mathematics - Optimization and Control
Druh dokumentu:	Working Paper
Popis:	We firstly propose the new stochastic gradient estimate of unbiasedness and minimized variance in this paper. Secondly, we propose the two algorithms: Algorithml and Algorithm2 which apply the new stochastic gradient estimate to modern stochastic conjugate gradient algorithms SCGA 7and CGVR 8. Then we prove that the proposed algorithms can obtain linearconvergence rate under assumptions of strong convexity and smoothness. Finally, numerical experiments show that the new stochastic gradient estimatecan reduce variance of stochastic gradient effectively. And our algorithms compared SCGA and CGVR can convergent faster in numerical experimentson ridge regression model. Comment: 17 pages, 3 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2306.00459 Zobrazit plný text záznamu View this record from Arxiv