A frequency-domain analysis of inexact gradient methods
| Autor: | Oran Gannot |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning General Mathematics 0211 other engineering and technologies MathematicsofComputing_NUMERICALANALYSIS 010103 numerical & computational mathematics 02 engineering and technology Systems and Control (eess.SY) 01 natural sciences Electrical Engineering and Systems Science - Systems and Control Machine Learning (cs.LG) Approximation error Convergence (routing) FOS: Mathematics FOS: Electrical engineering electronic engineering information engineering Applied mathematics Mathematics - Numerical Analysis 0101 mathematics Mathematics - Optimization and Control Mathematics 021103 operations research Numerical analysis Numerical Analysis (math.NA) Nonlinear system Rate of convergence Optimization and Control (math.OC) Frequency domain Gradient descent Convex function Software |
| Popis: | We study robustness properties of some iterative gradient-based methods for strongly convex functions, as well as for the larger class of functions with sector-bounded gradients, under a relative error model. Proofs of the corresponding convergence rates are based on frequency-domain criteria for the stability of nonlinear systems. Applications are given to inexact versions of gradient descent and the Triple Momentum Method. To further emphasize the usefulness of frequency-domain methods, we derive improved analytic bounds for the convergence rate of Nesterov's accelerated method (in the exact setting) on strongly convex functions. 42 pages; corrections and additional applications to accelerated methods. To appear in Mathematical Programming |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|