The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than $1/k^{2}$

Autor: Attouch, Hedy, Peypouquet, Juan
Rok vydání: 2015
Předmět:
Zdroj: SIAM Journal on Optimization 26 (2016), no. 3, 1824-1834
Druh dokumentu: Working Paper
DOI: 10.1137/15M1046095
Popis: The {\it forward-backward algorithm} is a powerful tool for solving optimization problems with a {\it additively separable} and {\it smooth} + {\it nonsmooth} structure. In the convex setting, a simple but ingenious acceleration scheme developed by Nesterov has been proved useful to improve the theoretical rate of convergence for the function values from the standard $\mathcal O(k^{-1})$ down to $\mathcal O(k^{-2})$. In this short paper, we prove that the rate of convergence of a slight variant of Nesterov's accelerated forward-backward method, which produces {\it convergent} sequences, is actually $o(k^{-2})$, rather than $\mathcal O(k^{-2})$. Our arguments rely on the connection between this algorithm and a second-order differential inclusion with vanishing damping.
Databáze: arXiv