Decomposition Methods with Deep Corrections for Reinforcement Learning
| Autor: | Mykel J. Kochenderfer, Kikuo Fujimura, Alireza Nakhaei, Maxime Bouton, Kyle D. Julian |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Mathematical optimization Artificial neural network Computer Science - Artificial Intelligence Computer science 05 social sciences 02 engineering and technology Machine Learning (cs.LG) Artificial Intelligence (cs.AI) Artificial Intelligence 0502 economics and business 0202 electrical engineering electronic engineering information engineering Reinforcement learning 050206 economic theory 020201 artificial intelligence & image processing Decomposition method (constraint satisfaction) |
| DOI: | 10.48550/arxiv.1802.01772 |
| Popis: | Decomposition methods have been proposed to approximate solutions to large sequential decision making problems. In contexts where an agent interacts with multiple entities, utility decomposition can be used to separate the global objective into local tasks considering each individual entity independently. An arbitrator is then responsible for combining the individual utilities and selecting an action in real time to solve the global problem. Although these techniques can perform well empirically, they rely on strong assumptions of independence between the local tasks and sacrifice the optimality of the global solution. This paper proposes an approach that improves upon such approximate solutions by learning a correction term represented by a neural network. We demonstrate this approach on a fisheries management problem where multiple boats must coordinate to maximize their catch over time as well as on a pedestrian avoidance problem for autonomous driving. In each problem, decomposition methods can scale to multiple boats or pedestrians by using strategies involving one entity. We verify empirically that the proposed correction method significantly improves the decomposition method and outperforms a policy trained on the full scale problem without utility decomposition. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|