Towards feature selection in actor-critic algorithms
| Autor: | Rohanimanesh, Khashayar, Roy, Nicholas, Tedrake, Russell Louis |
|---|---|
| Přispěvatelé: | Massachusetts Institute of Technology. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science, Tedrake, Russell Louis, Roy, Nicholas |
| Jazyk: | angličtina |
| Rok vydání: | 2009 |
| Předmět: | |
| Zdroj: | MIT web domain |
| Popis: | URL to paper listed on conference page Choosing features for the critic in actor-critic algorithms with function approximation is known to be a challenge. Too few critic features can lead to degeneracy of the actor gradient, and too many features may lead to slower convergence of the learner. In this paper, we show that a wellstudied class of actor policies satisfy the known requirements for convergence when the actor features are selected carefully. We demonstrate that two popular representations for value methods - the barycentric interpolators and the graph Laplacian proto-value functions - can be used to represent the actor in order to satisfy these conditions. A consequence of this work is a generalization of the proto-value function methods to the continuous action actor-critic domain. Finally, we analyze the performance of this approach using a simulation of a torque-limited inverted pendulum. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|