Nonstrict Hierarchical Reinforcement Learning for Interactive Systems and Robots
| Autor: | Nina Dethlefs, Heriberto Cuayáhuitl, Ivana Kruijff-Korbayová |
|---|---|
| Rok vydání: | 2014 |
| Předmět: |
business.industry
Computer science G710 Speech and Natural Language Processing G700 Artificial Intelligence Robot learning Human–robot interaction Human-Computer Interaction Function approximation Artificial Intelligence Bellman equation Scalability State space Robot Reinforcement learning Artificial intelligence G760 Machine Learning business |
| Zdroj: | ACM Transactions on Interactive Intelligent Systems. 4:1-30 |
| ISSN: | 2160-6463 2160-6455 |
| DOI: | 10.1145/2659003 |
| Popis: | Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|