| Autor: |
Soubki, Adil, Murzaku, John, Jordehi, Arash Yousefi, Zeng, Peter, Markowska, Magdalena, Mirroshandel, Seyed Abolghasem, Rambow, Owen |
| Rok vydání: |
2024 |
| Předmět: |
|
| Zdroj: |
ACL 2024 Findings |
| Druh dokumentu: |
Working Paper |
| Popis: |
Evaluating the theory of mind (ToM) capabilities of language models (LMs) has recently received a great deal of attention. However, many existing benchmarks rely on synthetic data, which risks misaligning the resulting experiments with human behavior. We introduce the first ToM dataset based on naturally occurring spoken dialogs, Common-ToM, and show that LMs struggle to demonstrate ToM. We then show that integrating a simple, explicit representation of beliefs improves LM performance on Common-ToM. |
| Databáze: |
arXiv |
| Externí odkaz: |
|
