A Universal Empirical Dynamic Programming Algorithm for Continuous State MDPs
| Autor: | William B. Haskell, Rahul Jain, Pengqian Yu, Hiteshi Sharma |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
0209 industrial biotechnology
Function space Computer science Approximation property Nonparametric statistics Probabilistic logic Approximation algorithm Stochastic dominance Basis function 02 engineering and technology Computer Science Applications 020901 industrial engineering & automation Operator (computer programming) Function approximation Control and Systems Engineering Applied mathematics Markov decision process Electrical and Electronic Engineering Parametric equation Parametric statistics Reproducing kernel Hilbert space |
| Zdroj: | IEEE Transactions on Automatic Control. 65:115-129 |
| ISSN: | 2334-3303 0018-9286 |
| DOI: | 10.1109/tac.2019.2907414 |
| Popis: | We propose universal randomized function approximation-based empirical value learning (EVL) algorithms for Markov decision processes. The “empirical” nature comes from each iteration being done empirically from samples available from simulations of the next state. This makes the Bellman operator a random operator. A parametric and a nonparametric method for function approximation using a parametric function space and a reproducing kernel Hilbert space respectively are then combined with EVL. Both function spaces have the universal function approximation property. Basis functions are picked randomly. Convergence analysis is performed using a random operator framework with techniques from the theory of stochastic dominance. Finite time sample complexity bounds are derived for both universal approximate dynamic programming algorithms. Numerical experiments support the versatility and computational tractability of this approach. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|