| Autor: |
Lopez-Gazpio, I, Gaviria, J, García, P, Sanjurjo-González, H, Sanz, B, Zarranz, A, Maritxalar, M, Agirre, E |
| Předmět: |
|
| Zdroj: |
Logic Journal of the IGPL; Dec2024, Vol. 32 Issue 6, p1088-1101, 14p |
| Abstrakt: |
We present PhrasIS, a benchmark dataset composed of natural occurring Phras e pairs with I nference and S imilarity annotations for the evaluation of semantic representations. The described dataset fills the gap between word and sentence-level datasets, allowing to evaluate compositional models at a finer granularity than sentences. Contrary to other datasets, the phrase pairs are extracted from naturally occurring text in image captions and news headlines. All the text fragments have been annotated by experts following a rigorous process also described in the manuscript achieving high inter annotator agreement. In this work we analyse the dataset, showing the relation between inference labels and similarity scores. With 10K phrase pairs split in development and test, the dataset is an excellent benchmark for testing meaning representation systems. [ABSTRACT FROM AUTHOR] |
| Databáze: |
Complementary Index |
| Externí odkaz: |
|
