Building Earth Mover's Distance on Bilingual Word Embeddings for Machine Translation
| Autor: | Meng Zhang, Yang Liu, Huanbo Luan, Maosong Sun, Tatsuya Izuha, Jie Hao |
|---|---|
| Rok vydání: | 2016 |
| Předmět: | |
| Zdroj: | Proceedings of the AAAI Conference on Artificial Intelligence. 30 |
| ISSN: | 2374-3468 2159-5399 |
| DOI: | 10.1609/aaai.v30i1.10351 |
| Popis: | Following their monolingual counterparts, bilingual word embeddings are also on the rise. As a major application task, word translation has been relying on the nearest neighbor to connect embeddings cross-lingually. However, the nearest neighbor strategy suffers from its inherently local nature and fails to cope with variations in realistic bilingual word embeddings. Furthermore, it lacks a mechanism to deal with many-to-many mappings that often show up across languages. We introduce Earth Mover's Distance to this task by providing a natural formulation that translates words in a holistic fashion, addressing the limitations of the nearest neighbor. We further extend the formulation to a new task of identifying parallel sentences, which is useful for statistical machine translation systems, thereby expanding the application realm of bilingual word embeddings. We show encouraging performance on both tasks. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|