Using Visual Feature Space as a Pivot Across Languages
| Autor: | Vicente Ordonez, Ziyan Yang, Franck Dernoncourt, Leticia Pinto-Alva |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Machine translation
business.industry Computer science Feature vector Inference 02 engineering and technology 010501 environmental sciences computer.software_genre 01 natural sciences 0202 electrical engineering electronic engineering information engineering Leverage (statistics) 020201 artificial intelligence & image processing Artificial intelligence business computer Natural language processing 0105 earth and related environmental sciences |
| Zdroj: | EMNLP (Findings) |
| DOI: | 10.18653/v1/2020.findings-emnlp.328 |
| Popis: | Our work aims to leverage visual feature space to pass information across languages. We show that models trained to generate textual captions in more than one language conditioned on an input image can leverage their jointly trained feature space during inference to pivot across languages. We particularly demonstrate improved quality on a caption generated from an input image, by leveraging a caption in a second language. More importantly, we demonstrate that even without conditioning on any visual input, the model demonstrates to have learned implicitly to perform to some extent machine translation from one language to another in their shared visual feature space. We show results in German-English, and Japanese-English language pairs that pave the way for using the visual world to learn a common representation for language. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|