Mind the Context: The Impact of Contextualization in Neural Module Networks for Grounding Visual Referring Expressions

Autor: Arjun Akula, Spandana Gella, Keze Wang, Song-Chun Zhu, Siva Reddy

Publikováno v: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_________::c71cbf028ab2dd6271b31560beca0d06
https://doi.org/10.18653/v1/2021.emnlp-main.516

Words Aren’t Enough, Their Order Matters: On the Robustness of Grounding Visual Referring Expressions

Autor: Yaser Al-Onaizan, Arjun R. Akula, Spandana Gella, Song-Chun Zhu, Siva Reddy

Publikováno v: ACL

Visual referring expression recognition is a challenging task that requires natural language understanding in the context of an image. We critically examine RefCOCOg, a standard benchmark for this task, using a human study and show that 83.7% of test

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_________::2f3809de83b3f64b89539324a7cb655b
https://doi.org/10.18653/v1/2020.acl-main.586

Cross-lingual Visual Verb Sense Disambiguation

Autor: Desmond Elliott, Frank Keller, Spandana Gella

Publikováno v: Gella, S, Elliott, D & Keller, F 2019, Cross-lingual Visual Verb Sense Disambiguation . in J Burstein, C Doran & T Solorio (eds), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics . vol. 1, Minneapolis, Minnesota, pp. 1998–2004, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, United States, 2/06/19 . https://doi.org/10.18653/v1/n19-1200
NAACL-HLT (1)

Recent work has shown that visual context improves cross-lingual sense disambiguation for nouns. We extend this line of work to the more challenging task of cross-lingual verb sense disambiguation, introducing the MultiSense dataset of 9,504 images a

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::5846e9e5ecf9495e5e3b22b8122b4d42
https://www.pure.ed.ac.uk/ws/files/82479457/Cross_lingual_Visual_Verb_GELLA_DoA220219_AFV.pdf

An Evaluation of Image-Based Verb Prediction Models against Human Eye-Tracking Data

Autor: Spandana Gella, Frank Keller

Publikováno v: NAACL-HLT (2)
Gella, S & Keller, F 2018, An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data . in The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . New Orleans, Louisiana, pp. 758-763, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, United States, 1/06/18 . https://doi.org/10.18653/v1/N18-2119

Recent research in language and vision has developed models for predicting and disambiguating verbs from images. Here, we ask whether the predictions made by such models correspond to human intuitions about visual verbs. We show that the image region

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::11d80108cefd15247596904cdbd84546
https://doi.org/10.18653/v1/n18-2119

Disambiguating Visual Verbs

Autor: Frank Keller, Mirella Lapata, Spandana Gella

Publikováno v: IEEE Transactions on Pattern Analysis and Machine Intelligence
Gella, S, Keller, F & Lapata, M 2019, ' Disambiguating Visual Verbs ', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 2, pp. 311-322 . https://doi.org/10.1109/TPAMI.2017.2786699

In this article, we introduce a new task, visual sense disambiguation for verbs: given an image and a verb, assign the correct sense of the verb, i.e., the one that describes the action depicted in the image. Just as textual word sense disambiguation

Externí odkaz: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::502e0df1f2320b34ffe50b576fb322f9