Assessing Reference-Free Peer Evaluation for Machine Translation
| Autor: | George Foster, Sweta Agrawal, Colin Cherry, Markus Freitag |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Computation and Language Theoretical computer science Machine translation Computer science 02 engineering and technology computer.software_genre Reference free 03 medical and health sciences 0302 clinical medicine Metric (mathematics) Scalability 030221 ophthalmology & optometry 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing State (computer science) Computation and Language (cs.CL) Scaling computer BLEU Peer evaluation |
| Zdroj: | NAACL-HLT |
| Popis: | Reference-free evaluation has the potential to make machine translation evaluation substantially more scalable, allowing us to pivot easily to new languages or domains. It has been recently shown that the probabilities given by a large, multilingual model can achieve state of the art results when used as a reference-free metric. We experiment with various modifications to this model and demonstrate that by scaling it up we can match the performance of BLEU. We analyze various potential weaknesses of the approach and find that it is surprisingly robust and likely to offer reasonable performance across a broad spectrum of domains and different system qualities. NAACL 2021 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|