Belief Revision based Caption Re-ranker with Visual Semantic Information

Autor:	Sabir, Ahmed, Moreno-Noguer, Francesc, Madhyastha, Pranava, Padró, Lluís
Rok vydání:	2022
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	In this work, we focus on improving the captions generated by image-caption generation systems. We propose a novel re-ranking approach that leverages visual-semantic measures to identify the ideal caption that maximally captures the visual information in the image. Our re-ranker utilizes the Belief Revision framework (Blok et al., 2003) to calibrate the original likelihood of the top-n captions by explicitly exploiting the semantic relatedness between the depicted caption and the visual context. Our experiments demonstrate the utility of our approach, where we observe that our re-ranker can enhance the performance of a typical image-captioning system without the necessity of any additional training or fine-tuning. Comment: COLING 2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2209.08163 Zobrazit plný text záznamu View this record from Arxiv