Unbabel's Submission to the WMT2019 APE Shared Task: BERT-based Encoder-Decoder for Automatic Post-Editing
Autor: | Gonçalo M. Correia, António V. Lopes, Jonay Trénous, M. Amin Farajian, André F. T. Martins |
---|---|
Jazyk: | angličtina |
Rok vydání: | 2019 |
Předmět: |
FOS: Computer and information sciences
Training set Computer Science - Computation and Language Machine translation Computer science Speech recognition computer.software_genre Task (project management) Factor (programming language) Encoder decoder computer Encoder Computation and Language (cs.CL) computer.programming_language |
Zdroj: | WMT (3) |
Popis: | This paper describes Unbabel's submission to the WMT2019 APE Shared Task for the English-German language pair. Following the recent rise of large, powerful, pre-trained models, we adapt the BERT pretrained model to perform Automatic Post-Editing in an encoder-decoder framework. Analogously to dual-encoder architectures we develop a BERT-based encoder-decoder (BED) model in which a single pretrained BERT encoder receives both the source src and machine translation tgt strings. Furthermore, we explore a conservativeness factor to constrain the APE system to perform fewer edits. As the official results show, when trained on a weighted combination of in-domain and artificial training data, our BED system with the conservativeness penalty improves significantly the translations of a strong Neural Machine Translation system by $-0.78$ and $+1.23$ in terms of TER and BLEU, respectively. Finally, our submission achieves a new state-of-the-art, ex-aequo, in English-German APE of NMT. Updated sections 2.2 and 4 |
Databáze: | OpenAIRE |
Externí odkaz: |
načítá se...