Evaluating the Robustness of Adverse Drug Event Classification Models Using Templates

Autor:	MacPhail, Dorothea, Harbecke, David, Raithel, Lisa, Möller, Sebastian
Rok vydání:	2024
Předmět:	Computer Science - Computation and Language Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	An adverse drug effect (ADE) is any harmful event resulting from medical drug treatment. Despite their importance, ADEs are often under-reported in official channels. Some research has therefore turned to detecting discussions of ADEs in social media. Impressive results have been achieved in various attempts to detect ADEs. In a high-stakes domain such as medicine, however, an in-depth evaluation of a model's abilities is crucial. We address the issue of thorough performance evaluation in English-language ADE detection with hand-crafted templates for four capabilities: Temporal order, negation, sentiment, and beneficial effect. We find that models with similar performance on held-out test sets have varying results on these capabilities. Comment: Accepted at BioNLP 2024 and Shared Tasks (ACL Workshop)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2407.02432 Zobrazit plný text záznamu View this record from Arxiv