Hierarchical Reinforcement Learning with Options and United Neural Network Approximation
| Autor: | Aleksandr I. Panov, Vadim Kuzmin |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
0209 industrial biotechnology
Hierarchy (mathematics) Artificial neural network business.industry Computer science Q-learning 02 engineering and technology Convolutional neural network 020901 industrial engineering & automation 0202 electrical engineering electronic engineering information engineering Reinforcement learning Robot 020201 artificial intelligence & image processing Artificial intelligence business Curse of dimensionality Abstraction (linguistics) |
| Zdroj: | Advances in Intelligent Systems and Computing ISBN: 9783030018177 |
| DOI: | 10.1007/978-3-030-01818-4_45 |
| Popis: | The “curse of dimensionality” and environments with sparse delayed rewards are one of the main challenges in reinforcement learning (RL). To tackle these problems we can use hierarchical reinforcement learning (HRL) that provides abstraction both on actions and states of the environment. This work proposes an algorithm that combines hierarchical approach for RL and the ability of neural networks to serve as universal function approximators. To perform the hierarchy of actions the options framework is used which main idea is to utilize macro-actions (the sequence of simpler actions). State of the environment is the input to a convolutional neural network that plays a role of Q-function estimating the utility of every possible action and skill in the given state. We learn each option separately using different neural networks and then combine result into one architecture with top-level approximator. We compare the performance of the proposed algorithm with the deep Q-network algorithm (DQN) in the environment where the aim of the magnet-arm robot is to build a tower from bricks. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|