Data and systems for medication-related text classification and concept normalization from Twitter: Insights from the Social Media Mining for Health (SMM4H) 2017 shared task
| Autor: | Jasper Friedrichs, Abeed Sarker, Filip Ginter, Maksim Belousov, Saif M. Mohammad, Ramakanth Kavuluru, Debanjan Mahata, Berry de Bruijn, Tung Tran, Anthony Rios, Graciela Gonzalez-Hernandez, Sifei Han, Kai Hakala, Svetlana Kiritchenko, Goran Nenadic, Farrokh Mehryary |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2018 |
| Předmět: |
Normalization (statistics)
Support Vector Machine Drug-Related Side Effects and Adverse Reactions 020205 medical informatics Computer science social media Health Informatics 02 engineering and technology text mining Research and Applications computer.software_genre Convolutional neural network 03 medical and health sciences Pharmacovigilance 0302 clinical medicine Text mining Text processing Social media mining Machine learning 0202 electrical engineering electronic engineering information engineering Data Mining Humans Social media 030212 general & internal medicine natural language processing ta515 business.industry Natural language processing Identifier Support vector machine machine learning pharmacovigilance Neural Networks Computer Artificial intelligence business computer |
| Zdroj: | Sarkerb, A, Belousov, M, Friedrichs, J, Hakala, K, Kiritchenko, S, Mehryary, F, Han, S, Tran, T, Rios, A, Kavuluru, R, de Bruijn, B, Ginter, F, Mahata, D, Mohammad, S M, Nenadic, G & Gonzalez-Hernandez, G 2018, ' Data and systems for medication-related text classification and concept normalization from Twitter: Insights from the Social Media Mining for Health (SMM4H) 2017 shared task ', Journal of the American Medical Informatics Association, vol. 25, no. 10, pp. 1274-1283 . https://doi.org/10.1093/jamia/ocy114 Journal of the American Medical Informatics Association : JAMIA |
| Popis: | Objective: We executed the Social Media Mining for Health (SMM4H) 2017 shared tasks to enable the community-driven development and large-scale evaluation of automatic text processing methods for the classification and normalization of health-related text from social media. An additional objective was to publicly release manually annotated data. Materials and Methods: We organized 3 independent subtasks: automatic classification of self-reports of 1) adverse drug reactions (ADRs) and 2) medication consumption, from medication-mentioning tweets, and 3) normalization of ADR expressions. Training data consisted of 15 717 annotated tweets for (1), 10 260 for (2), and 6650 ADR phrases and identifiers for (3); and exhibited typical properties of social-media-based health-related texts. Systems were evaluated using 9961, 7513, and 2500 instances for the 3 subtasks, respectively. We evaluated performances of classes of methods and ensembles of system combinations following the shared tasks. Results: Among 55 system runs, the best system scores for the 3 subtasks were 0.435 (ADR class F1-score) for subtask-1, 0.693 (micro-averaged F1-score over two classes) for subtask-2, and 88.5% (accuracy) for subtask-3. Ensembles of system combinations obtained best scores of 0.476, 0.702, and 88.7%, outperforming individual systems. Discussion: Among individual systems, support vector machines and convolutional neural networks showed high performance. Performance gains achieved by ensembles of system combinations suggest that such strategies may be suitable for operational systems relying on difficult text classification tasks (eg, subtask-1). Conclusions: Data imbalance and lack of context remain challenges for natural language processing of social media text. Annotated data from the shared task have been made available as reference standards for future studies (http://dx.doi.org/10.17632/rxwfb3tysd.1). gold access |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|