Koopman Spectrum Nonlinear Regulators and Efficient Online Learning
| Autor: | Ohnishi, Motoya, Ishikawa, Isao, Lowrey, Kendall, Ikeda, Masahiro, Kakade, Sham, Kawahara, Yoshinobu |
|---|---|
| Rok vydání: | 2021 |
| Předmět: | |
| Zdroj: | Transactions on Machine Learning Research (https://openreview.net/forum?id=thfoUZugvS), 2024 |
| Druh dokumentu: | Working Paper |
| Popis: | Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, behaviors with sudden accelerations that waste energy and lack predictability. In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics. This induces a broader class of dynamical behaviors that evolve over stable manifolds such as nonlinear oscillators, closed loops, and smooth movements. We demonstrate that some dynamics characterizations that are not possible with a cumulative cost are feasible in this paradigm, which generalizes the classical eigenstructure and pole assignments to nonlinear decision making. Moreover, we present a sample efficient online learning algorithm for our problem that enjoys a sub-linear regret bound under some structural assumptions. Comment: 41 pages, 21 figures |
| Databáze: | arXiv |
| Externí odkaz: |
|