ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems

Autor:	Unanue, Inigo Jauregi, Borzeshi, Ehsan Zare, Esmaili, Nazanin, Piccardi, Massimo
Rok vydání:	2019
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	Regularization of neural machine translation is still a significant problem, especially in low-resource settings. To mollify this problem, we propose regressing word embeddings (ReWE) as a new regularization technique in a system that is jointly trained to predict the next word in the translation (categorical value) and its word embedding (continuous value). Such a joint training allows the proposed system to learn the distributional properties represented by the word embeddings, empirically improving the generalization to unseen sentences. Experiments over three translation datasets have showed a consistent improvement over a strong baseline, ranging between 0.91 and 2.54 BLEU points, and also a marked improvement over a state-of-the-art system. Comment: Accepted at NAACL-HLT 2019
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/1904.02461 Zobrazit plný text záznamu View this record from Arxiv