Fast reinforcement learning with generalized policy updates
| Autor: | Diana Borsa, David Silver, Shaobo Hou, Andre Barreto, Doina Precup |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Multidisciplinary
Speedup Exploit Computer science business.industry Deep learning 010501 environmental sciences Decision problem 01 natural sciences 03 medical and health sciences 0302 clinical medicine Obstacle Reinforcement learning Leverage (statistics) Artificial intelligence Linear combination business 030217 neurology & neurosurgery 0105 earth and related environmental sciences Colloquium on the Science of Deep Learning |
| Zdroj: | Proc Natl Acad Sci U S A |
| ISSN: | 1091-6490 |
| Popis: | The combination of reinforcement learning with deep learning is a promising approach to tackle important sequential decision-making problems that are currently intractable. One obstacle to overcome is the amount of data needed by learning systems of this type. In this article, we propose to address this issue through a divide-and-conquer approach. We argue that complex decision problems can be naturally decomposed into multiple tasks that unfold in sequence or in parallel. By associating each task with a reward function, this problem decomposition can be seamlessly accommodated within the standard reinforcement-learning formalism. The specific way we do so is through a generalization of two fundamental operations in reinforcement learning: policy improvement and policy evaluation. The generalized version of these operations allow one to leverage the solution of some tasks to speed up the solution of others. If the reward function of a task can be well approximated as a linear combination of the reward functions of tasks previously solved, we can reduce a reinforcement-learning problem to a simpler linear regression. When this is not the case, the agent can still exploit the task solutions by using them to interact with and learn about the environment. Both strategies considerably reduce the amount of data needed to solve a reinforcement-learning problem. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|