ParaCrawl: Web-Scale Acquisition of Parallel Corpora
| Autor: | Sergio Ortiz Rojas, Marek Strelec, Amir Kamran, Pinzhen Chen, Jaume Zaragoza, William Waites, Kenneth Heafield, Marta Bañón, Philipp Koehn, Hieu Hoang, Leopoldo Pla Sempere, Brian Thompson, Dion Wiggins, Elsa Sarrías, Faheem Kirefu, Gema Ramírez-Sánchez, Mikel L. Forcada, Barry Haddow, Miquel Esplà-Gomis |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
050101 languages & linguistics
Machine translation business.industry Computer science 05 social sciences InformationSystems_INFORMATIONSTORAGEANDRETRIEVAL 02 engineering and technology Crawling computer.software_genre Parallel corpora 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing 0501 psychology and cognitive sciences Artificial intelligence Scale (map) business computer Natural language processing Sentence |
| Zdroj: | Bañón, M, Chen, P, Haddow, B, Heafield, K, Hoang, H, Esplà-Gomis, M, Forcada, M, Kamran, A, Kirefu, F, Koehn, P, Ortiz-Rojas, S, Pla, L, Ramírez-Sánchez, G, Sarrías, E, Strelec, M, Thompson, B, Waites, W, Wiggins, D & Zaragoza, J 2020, ParaCrawl: Web-Scale Acquisition of Parallel Corpora . in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . pp. 4555–4567, 2020 Annual Conference of the Association for Computational Linguistics, Virtual conference, Washington, United States, 5/07/20 . https://doi.org/10.18653/v1/2020.acl-main.417 ACL |
| Popis: | We report on methods to create the largest publicly available parallel corpora by crawling the web, using open source software. We empirically compare alternative methods and publish benchmark data sets for sentence alignment and sentence pair filtering. We also describe the parallel corpora released and evaluate their quality and their usefulness to create machine translation systems. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|