Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite
| Autor: | Preslav Nakov, Shafiq Joty, Irina Temnikova, Prathyusha Jwalapuram |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Measure (data warehouse) Pronoun Computer Science - Machine Learning Computer Science - Computation and Language Machine translation Computer science business.industry Computer Science - Artificial Intelligence Anaphora (linguistics) 02 engineering and technology computer.software_genre Translation (geometry) Machine Learning (cs.LG) Artificial Intelligence (cs.AI) 020204 information systems 0202 electrical engineering electronic engineering information engineering Test suite 020201 artificial intelligence & image processing Artificial intelligence business Computation and Language (cs.CL) computer Natural language processing BLEU |
| Zdroj: | EMNLP/IJCNLP (1) |
| Popis: | The ongoing neural revolution in machine translation has made it easier to model larger contexts beyond the sentence-level, which can potentially help resolve some discourse-level ambiguities such as pronominal anaphora, thus enabling better translations. Unfortunately, even when the resulting improvements are seen as substantial by humans, they remain virtually unnoticed by traditional automatic evaluation measures like BLEU, as only a few words end up being affected. Thus, specialized evaluation measures are needed. With this aim in mind, we contribute an extensive, targeted dataset that can be used as a test suite for pronoun translation, covering multiple source languages and different pronoun errors drawn from real system translations, for English. We further propose an evaluation measure to differentiate good and bad pronoun translations. We also conduct a user study to report correlations with human judgments. Accepted at EMNLP 2019 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|