Zobrazeno 1 - 10
of 91
pro vyhledávání: '"Ye, Haishan"'
Variance reduction techniques are designed to decrease the sampling variance, thereby accelerating convergence rates of first-order (FO) and zeroth-order (ZO) optimization methods. However, in composite optimization problems, ZO methods encounter an
Externí odkaz:
http://arxiv.org/abs/2405.17761
This paper considers the distributed convex-concave minimax optimization under the second-order similarity. We propose stochastic variance-reduced optimistic gradient sliding (SVOGS) method, which takes the advantage of the finite-sum structure in th
Externí odkaz:
http://arxiv.org/abs/2405.16126
Anderson Acceleration Without Restart: A Novel Method with $n$-Step Super Quadratic Convergence Rate
In this paper, we propose a novel Anderson's acceleration method to solve nonlinear equations, which does \emph{not} require a restart strategy to achieve numerical stability. We propose the greedy and random versions of our algorithm. Specifically,
Externí odkaz:
http://arxiv.org/abs/2403.16734
Fine-tuning large language models (LLMs) with classic first-order optimizers entails prohibitive GPU memory due to the backpropagation process. Recent works have turned to zeroth-order optimizers for fine-tuning, which save substantial memory by usin
Externí odkaz:
http://arxiv.org/abs/2402.15173
Autor:
Ye, Haishan, Chang, Xiangyu
In this paper, we focus on the decentralized composite optimization for convex functions. Because of advantages such as robust to the network and no communication bottle-neck in the central server, the decentralized optimization has attracted much re
Externí odkaz:
http://arxiv.org/abs/2312.15845
Personalization aims to characterize individual preferences and is widely applied across many fields. However, conventional personalized methods operate in a centralized manner and potentially expose the raw data when pooling individual information.
Externí odkaz:
http://arxiv.org/abs/2310.14337
Autor:
Chen, Jun, Ye, Haishan, Wang, Mengmeng, Huang, Tianxin, Dai, Guang, Tsang, Ivor W., Liu, Yong
Publikováno v:
International Conference on Learning Representations, 2024
The conjugate gradient method is a crucial first-order optimization method that generally converges faster than the steepest descent method, and its computational cost is much lower than that of second-order methods. However, while various types of c
Externí odkaz:
http://arxiv.org/abs/2308.10547
Autor:
Ye, Haishan
The zeroth-order optimization has been widely used in machine learning applications. However, the theoretical study of the zeroth-order optimization focus on the algorithms which approximate (first-order) gradients using (zeroth-order) function value
Externí odkaz:
http://arxiv.org/abs/2308.00469
We study finite-sum distributed optimization problems involving a master node and $n-1$ local nodes under the popular $\delta$-similarity and $\mu$-strong convexity conditions. We propose two new algorithms, SVRS and AccSVRS, motivated by previous wo
Externí odkaz:
http://arxiv.org/abs/2304.07504
Autor:
Ye, Haishan, Chang, Xiangyu
In decentralized optimization, $m$ agents form a network and only communicate with their neighbors, which gives advantages in data ownership, privacy, and scalability. At the same time, decentralized stochastic gradient descent (\texttt{SGD}) methods
Externí odkaz:
http://arxiv.org/abs/2212.05273