Hedge algorithm and Dual Averaging schemes
| Autor: | Michael Bürgisser, Michel Baes |
|---|---|
| Rok vydání: | 2012 |
| Předmět: |
Scheme (programming language)
Mathematical optimization 021103 operations research General Mathematics 0211 other engineering and technologies 0207 environmental engineering Context (language use) 02 engineering and technology Management Science and Operations Research Hedge (linguistics) Interpretation (model theory) Dual (category theory) Optimization and Control (math.OC) Convex optimization Convergence (routing) FOS: Mathematics A priori and a posteriori 020701 environmental engineering Mathematics - Optimization and Control Algorithm computer Software Mathematics computer.programming_language |
| Zdroj: | Mathematical Methods of Operations Research. 77:279-289 |
| ISSN: | 1432-5217 1432-2994 |
| DOI: | 10.1007/s00186-012-0418-1 |
| Popis: | We show that the Hedge algorithm, a method that is widely used in Machine Learning, can be interpreted as a particular instance of Dual Averaging schemes, which have recently been introduced by Nesterov for regret minimization. Based on this interpretation, we establish three alternative methods of the Hedge algorithm: one in the form of the original method, but with optimal parameters, one that requires less a priori information, and one that is better adapted to the context of the Hedge algorithm. All our modified methods have convergence results that are better or at least as good as the performance guarantees of the vanilla method. In numerical experiments, our methods significantly outperform the original scheme. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full text from SpringerLink