A Vietnamese Dataset for Evaluating Machine Reading Comprehension
| Autor: | Anh Hoang-Tu Nguyen, Kiet Van Nguyen, Vu Duc Nguyen, Ngan Luu-Thuy Nguyen |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Matching (statistics)
Computer science Process (engineering) business.industry First language Vietnamese 02 engineering and technology computer.software_genre language.human_language Task (project management) Comprehension 03 medical and health sciences 0302 clinical medicine Benchmark (surveying) 030221 ophthalmology & optometry 0202 electrical engineering electronic engineering information engineering Question answering language 020201 artificial intelligence & image processing Artificial intelligence business computer Natural language processing |
| Zdroj: | COLING |
| DOI: | 10.18653/v1/2020.coling-main.233 |
| Popis: | Over 97 million inhabitants speak Vietnamese as the native language in the world. However, there are few research studies on machine reading comprehension (MRC) in Vietnamese, the task of understanding a document or text, and answering questions related to it. Due to the lack of benchmark datasets for Vietnamese, we present the Vietnamese Question Answering Dataset (UIT-ViQuAD), a new dataset for the low-resource language as Vietnamese to evaluate MRC models. This dataset comprises over 23,000 human-generated question-answer pairs based on 5,109 passages of 174 Vietnamese articles from Wikipedia. In particular, we propose a new process of dataset creation for Vietnamese MRC. Our in-depth analyses illustrate that our dataset requires abilities beyond simple reasoning like word matching and demands complicate reasoning such as single-sentence and multiple-sentence inferences. Besides, we conduct experiments on state-of-the-art MRC methods in English and Chinese as the first experimental models on UIT-ViQuAD, which will be compared to further models. We also estimate human performances on the dataset and compare it to the experimental results of several powerful machine models. As a result, the substantial differences between humans and the best model performances on the dataset indicate that improvements can be explored on UIT-ViQuAD through future research. Our dataset is freely available to encourage the research community to overcome challenges in Vietnamese MRC. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|