Input-Output Stability of Gradient Descent: A Discrete-Time Passivity-Based Approach

Autor:	Moalemi, Sepehr, Forbes, James Richard
Rok vydání:	2024
Předmět:	Mathematics - Optimization and Control Electrical Engineering and Systems Science - Systems and Control
Druh dokumentu:	Working Paper
Popis:	This paper presents a discrete-time passivity-based analysis of the gradient descent method for a class of functions with sector-bounded gradients. Using a loop transformation, it is shown that the gradient descent method can be interpreted as a passive controller in negative feedback with a very strictly passive system. The passivity theorem is then used to guarantee input-output stability, as well as the global convergence, of the gradient descent method. Furthermore, provided that the lower and upper sector bounds are not equal, the input-output stability of the gradient descent method is guaranteed using the weak passivity theorem for a larger choice of step size. Finally, to demonstrate the utility of this passivity-based analysis, a new variation of the gradient descent method with variable step size is proposed by gain-scheduling the input and output of the gradient. Comment: Submitted to IEEE Control Systems Letters (L-CSS)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2411.16074 Zobrazit plný text záznamu View this record from Arxiv