CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation
| Autor: | Nishant Kambhatla, Logan Born, Anoop Sarkar |
|---|---|
| Rok vydání: | 2022 |
| Předmět: | |
| DOI: | 10.48550/arxiv.2204.00665 |
| Popis: | We propose a novel data-augmentation technique for neural machine translation based on ROT-$k$ ciphertexts. ROT-$k$ is a simple letter substitution cipher that replaces a letter in the plaintext with the $k$th letter after it in the alphabet. We first generate multiple ROT-$k$ ciphertexts using different values of $k$ for the plaintext which is the source side of the parallel data. We then leverage this enciphered training data along with the original parallel data via multi-source training to improve neural machine translation. Our method, CipherDAug, uses a co-regularization-inspired training procedure, requires no external data sources other than the original training data, and uses a standard Transformer to outperform strong data augmentation techniques on several datasets by a significant margin. This technique combines easily with existing approaches to data augmentation, and yields particularly strong results in low-resource settings. Comment: ACL 2022 Main Conf. camera ready version |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|