Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation
| Autor: | Wanrong Zhu, Kazoo Sone, Xin Wang, An Yan, Sugato Basu, William Yang Wang, Tsu-Jui Fu, Pradyumna Narayana |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Language understanding Computer Science - Computation and Language Computer Science - Artificial Intelligence Computer science Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition Style (sociolinguistics) Task (project management) Artificial Intelligence (cs.AI) Human–computer interaction Test set Transfer (computing) Leverage (statistics) Baseline (configuration management) Computation and Language (cs.CL) Natural language |
| Zdroj: | EACL Scopus-Elsevier |
| Popis: | One of the most challenging topics in Natural Language Processing (NLP) is visually-grounded language understanding and reasoning. Outdoor vision-and-language navigation (VLN) is such a task where an agent follows natural language instructions and navigates a real-life urban environment. Due to the lack of human-annotated instructions that illustrate intricate urban scenes, outdoor VLN remains a challenging task to solve. This paper introduces a Multimodal Text Style Transfer (MTST) learning approach and leverages external multimodal resources to mitigate data scarcity in outdoor navigation tasks. We first enrich the navigation data by transferring the style of the instructions generated by Google Maps API, then pre-train the navigator with the augmented external outdoor navigation dataset. Experimental results show that our MTST learning approach is model-agnostic, and our MTST approach significantly outperforms the baseline models on the outdoor VLN task, improving task completion rate by 8.7% relatively on the test set. Comment: EACL 2021 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|