Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention

Autor: Zhang, B., Titov, I., Sennrich, R., Inui, K., Jiang, J., Ng, V., Wan, X.

Publikováno v: Zhang, B, Titov, I & Sennrich, R 2019, Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention . in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing . pp. 898–909, 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Hong Kong, Hong Kong, 3/11/19 . https://doi.org/10.18653/v1/D19-1083
EMNLP/IJCNLP (1)
2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing: EMNLP-IJCNLP 2019 : proceedings of the conference : November 3-7, 2019, Hong Kong, China, 898-909
STARTPAGE=898;ENDPAGE=909;TITLE=2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing

The general trend in NLP is towards increasing model capacity and performance via deeper neural networks. However, simply stacking more layers of the popular Transformer architecture for machine translation results in poor convergence and high comput

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::79868d75404019d79cdb6da1dd22af31
https://www.pure.ed.ac.uk/ws/files/129171008/Improving_Deep_Transformer_ZHANG_DOA04112019_VOR_CC_BY.pdf

Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

Autor: Voita, E., Talbot, D., Moiseev, F., Sennrich, R., Titov, I., Korhonen, A., Traum, D., Màrquez, L.

Publikováno v: Voita, E, Talbot, D, Moiseev, F, Sennrich, R & Titov, I 2019, Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned . in A Korhonen, D Traum & L Màrquez (eds), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers) . Florence, Italy, pp. 5797–5808, 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28/07/19 . https://doi.org/10.18653/v1/P19-1580
The 57th Annual Meeting of the Association for Computational Linguistics: ACL 2019 : proceedings of the conference : July 28-August 2, 2019, Florence, Italy
The 57th Annual Meeting of the Association for Computational Linguistics
Voita, Elena; Talbot, David; Moiseev, Fedor; Sennrich, Rico; Titov, Ivan (2019). Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Italy, 28 July 2019-2 August 2019, 5797-5808.
ACL (1)

Multi-head self-attention is a key component of the Transformer, a state-of-the-art architecture for neural machine translation. In this work we evaluate the contribution made by individual attention heads in the encoder to the overall performance of

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::9b369726cc4cdd086f527026f45c2da0
https://www.pure.ed.ac.uk/ws/files/106651234/Analyzing_Multi_Head_VOITA_DoA020819_VoR_CC_BY.pdf

Context-Aware Monolingual Repair for Neural Machine Translation

Autor: Voita, E., Sennrich, R., Titov, I., Inui, K., Jiang, J., Ng, V., Wan, X.

Publikováno v: EMNLP/IJCNLP (1)
2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing: EMNLP-IJCNLP 2019 : proceedings of the conference : November 3-7, 2019, Hong Kong, China, 877-886
STARTPAGE=877;ENDPAGE=886;TITLE=2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing
Voita, E, Sennrich, R & Titov, I 2019, Context-Aware Monolingual Repair for Neural Machine Translation . in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing . Hong Kong, China, pp. 876-885, 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Hong Kong, Hong Kong, 3/11/19 . https://doi.org/10.18653/v1/D19-1081

Modern sentence-level NMT systems often produce plausible translations of isolated sentences. However, when put in context, these translations may end up being inconsistent with each other. We propose a monolingual DocRepair model to correct inconsis

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::5f6312c407eb028f804803b95f7df047
https://hdl.handle.net/11245.1/ee488c55-0148-497c-86da-26f5a1b65e29

The UZH system combination system for WMT 2011

Autor: Sennrich, R

Publikováno v: Sennrich, R 2011, The UZH system combination system for WMT 2011 . in Proceedings of the Sixth Workshop on Statistical Machine Translation . Edinburgh, Scotland, pp. 166-170, Sixth Workshop on Statistical Machine Translation, Edinburgh, United Kingdom, 30/07/11 . < http://dx.doi.org/10.5167/uzh-49037 >
Rico Sennrich

This paper describes the UZH system that was used for the WMT 2011 system combination shared task submission. We participated in the system combination task for the translation directions DE-EN and EN-DE. The system uses Moses as a backbone, with the

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::30934049c0ff88a643bb42da2b568d70
http://www.zora.uzh.ch/49037/

Word-aligned parallel text : a new resource for contrastive language studies

Autor: Volk, Martin, Göhring, A, Lehner, S, Rios, A, Sennrich, R, Uibo, H

Publikováno v: Volk, M, Göhring, A, Lehner, S, Rios, A, Sennrich, R & Uibo, H 2011, Word-aligned parallel text : a new resource for contrastive language studies . in Supporting Digital Humanities, Conference 2011 . Copenhagen, Denmark, Supporting Digital Humanities, Conference 2011, Copenhagen, Denmark, 17/11/11 .
University of Zurich

This paper describes the opportunities that arise from automatic word alignment for bilingual concordances and contrastive language studies. We introduce our parallel corpus of Alpine texts in French and German and our web-based alignment search syst

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=dedup_wf_001::6c5e42492d14581edb443b835aa82be6
https://hdl.handle.net/20.500.11820/a05431f1-01cf-4afa-9f9f-a0089bd8cbf3