xSIM++: An Improved Proxy to Bitext Mining Performance for Low-Resource Languages

Autor:	Chen, Mingda, Heffernan, Kevin, Çelebi, Onur, Mourachko, Alex, Schwenk, Holger
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	We introduce a new proxy score for evaluating bitext mining based on similarity in a multilingual embedding space: xSIM++. In comparison to xSIM, this improved proxy leverages rule-based approaches to extend English sentences in any evaluation set with synthetic, hard-to-distinguish examples which more closely mirror the scenarios we encounter during large-scale mining. We validate this proxy by running a significant number of bitext mining experiments for a set of low-resource languages, and subsequently train NMT systems on the mined data. In comparison to xSIM, we show that xSIM++ is better correlated with the downstream BLEU scores of translation systems trained on mined bitexts, providing a reliable proxy of bitext mining performance without needing to run expensive bitext mining pipelines. xSIM++ also reports performance for different error types, offering more fine-grained feedback for model development. Comment: The first two authors contributed equally; ACL 2023 short; Code and data are available at https://github.com/facebookresearch/LASER
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2306.12907 Zobrazit plný text záznamu View this record from Arxiv