Multimodal Aggregation Approach for Memory Vision-Voice Indoor Navigation with Meta-Learning
| Autor: | Yaoxian Song, Dongfang Liu, Changbin Yu, Liqi Yan |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Meta learning (computer science) Computer science Computer Vision and Pattern Recognition (cs.CV) Feature extraction Computer Science - Computer Vision and Pattern Recognition ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION 02 engineering and technology 010501 environmental sciences 01 natural sciences Visualization Human–computer interaction 0202 electrical engineering electronic engineering information engineering Task analysis RGB color model Robot 020201 artificial intelligence & image processing 0105 earth and related environmental sciences |
| Zdroj: | IROS |
| Popis: | Vision and voice are two vital keys for agents' interaction and learning. In this paper, we present a novel indoor navigation model called Memory Vision-Voice Indoor Navigation (MVV-IN), which receives voice commands and analyzes multimodal information of visual observation in order to enhance robots' environment understanding. We make use of single RGB images taken by a first-view monocular camera. We also apply a self-attention mechanism to keep the agent focusing on key areas. Memory is important for the agent to avoid repeating certain tasks unnecessarily and in order for it to adapt adequately to new scenes, therefore, we make use of meta-learning. We have experimented with various functional features extracted from visual observation. Comparative experiments prove that our methods outperform state-of-the-art baselines. Comment: 8 pages, 6 figures, 2 tables, accepted at 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020) |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|