Evaluating the Markov assumption in Markov Decision Processes for spoken dialogue management
| Autor: | Tim Paek, David Maxwell Chickering |
|---|---|
| Rok vydání: | 2006 |
| Předmět: |
Linguistics and Language
business.industry Computer science General Social Sciences Partially observable Markov decision process Markov process Library and Information Sciences Markov model Action selection Language and Linguistics Education symbols.namesake Robustness (computer science) symbols Reinforcement learning State space Markov property Artificial intelligence Markov decision process business |
| Zdroj: | Language Resources and Evaluation. 40:47-66 |
| ISSN: | 1572-8412 1574-020X |
| Popis: | The goal of dialogue management in a spoken dialogue system is to take actions based on observations and inferred beliefs. To ensure that the actions optimize the performance or robustness of the system, researchers have turned to reinforcement learning methods to learn policies for action selection. To derive an optimal policy from data, the dynamics of the system is often represented as a Markov Decision Process (MDP), which assumes that the state of the dialogue depends only on the previous state and action. In this article, we investigate whether constraining the state space by the Markov assumption, especially when the structure of the state space may be unknown, truly affords the highest reward. In simulation experiments conducted in the context of a dialogue system for interacting with a speech-enabled web browser, models under the Markov assumption did not perform as well as an alternative model which classifies the total reward with accumulating features. We discuss the implications of the study as well as its limitations. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full text from SpringerLink