LCP-dropout: Compression-based Multiple Subword Segmentation for Neural Machine Translation
| Autor: | Nonaka, Keita, Yamanouchi, Kazutaka, I, Tomohiro, Okita, Tsuyoshi, Shimada, Kazutaka, Sakamoto, Hiroshi |
|---|---|
| Rok vydání: | 2022 |
| Předmět: | |
| Zdroj: | Electronics 11(7), Article number 1014, 2022 |
| Druh dokumentu: | Working Paper |
| DOI: | 10.3390/electronics11071014 |
| Popis: | In this study, we propose a simple and effective preprocessing method for subword segmentation based on a data compression algorithm. Compression-based subword segmentation has recently attracted significant attention as a preprocessing method for training data in Neural Machine Translation. Among them, BPE/BPE-dropout is one of the fastest and most effective method compared to conventional approaches. However, compression-based approach has a drawback in that generating multiple segmentations is difficult due to the determinism. To overcome this difficulty, we focus on a probabilistic string algorithm, called locally-consistent parsing (LCP), that has been applied to achieve optimum compression. Employing the probabilistic mechanism of LCP, we propose LCP-dropout for multiple subword segmentation that improves BPE/BPE-dropout, and show that it outperforms various baselines in learning from especially small training data. Comment: 12 pages |
| Databáze: | arXiv |
| Externí odkaz: |
|