Variational Dynamic Programming for Stochastic Optimal Control
Autor: | Lambert, Marc, Bach, Francis, Bonnabel, Silvère |
---|---|
Rok vydání: | 2024 |
Předmět: | |
Zdroj: | 2024 Conference on Decision and Control, Dec 2024, Milano, Italy |
Druh dokumentu: | Working Paper |
Popis: | We consider the problem of stochastic optimal control, where the state-feedback control policies take the form of a probability distribution and where a penalty on the entropy is added. By viewing the cost function as a Kullback- Leibler (KL) divergence between two joint distributions, we bring the tools from variational inference to bear on our optimal control problem. This allows for deriving a dynamic programming principle, where the value function is defined as a KL divergence again. We then resort to Gaussian distributions to approximate the control policies and apply the theory to control affine nonlinear systems with quadratic costs. This results in closed-form recursive updates, which generalize LQR control and the backward Riccati equation. We illustrate this novel method on the simple problem of stabilizing an inverted pendulum. |
Databáze: | arXiv |
Externí odkaz: |
načítá se...