Towards Human-Centred Explainability Benchmarks For Text Classification

Autor:	Schlegel, Viktor, Mendez-Guzman, Erick, Batista-Navarro, Riza
Rok vydání:	2022
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	Progress on many Natural Language Processing (NLP) tasks, such as text classification, is driven by objective, reproducible and scalable evaluation via publicly available benchmarks. However, these are not always representative of real-world scenarios where text classifiers are employed, such as sentiment analysis or misinformation detection. In this position paper, we put forward two points that aim to alleviate this problem. First, we propose to extend text classification benchmarks to evaluate the explainability of text classifiers. We review challenges associated with objectively evaluating the capabilities to produce valid explanations which leads us to the second main point: We propose to ground these benchmarks in human-centred applications, for example by using social media, gamification or to learn explainability metrics from human judgements. Comment: Accepted at NeatClass @ ICSWSM 2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2211.05452 Zobrazit plný text záznamu View this record from Arxiv