Exploiting Rules to Enhance Machine Learning in Extracting Information From Multi-Institutional Prostate Pathology Reports
| Autor: | Amir M. Tahmasebi, Scott F. Thompson, Kevin S. Hughes, Peter Prinsen, Lance Mynderse, Regina Barzilay, Enrico Santus, Samuel Coons, Clara Li, Conor R. Lanahan, Adam Yala, Tal Schuster |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Male
Support Vector Machine Computer science business.industry Prostate General Medicine Machine learning computer.software_genre United States Machine Learning 03 medical and health sciences 0302 clinical medicine Logistic Models 030220 oncology & carcinogenesis Humans 030212 general & internal medicine Artificial intelligence business computer Algorithms |
| Zdroj: | JCO clinical cancer informatics. 4 |
| ISSN: | 2473-4276 |
| Popis: | PURPOSE Literature on clinical note mining has highlighted the superiority of machine learning (ML) over hand-crafted rules. Nevertheless, most studies assume the availability of large training sets, which is rarely the case. For this reason, in the clinical setting, rules are still common. We suggest 2 methods to leverage the knowledge encoded in pre-existing rules to inform ML decisions and obtain high performance, even with scarce annotations. METHODS We collected 501 prostate pathology reports from 6 American hospitals. Reports were split into 2,711 core segments, annotated with 20 attributes describing the histology, grade, extension, and location of tumors. The data set was split by institutions to generate a cross-institutional evaluation setting. We assessed 4 systems, namely a rule-based approach, an ML model, and 2 hybrid systems integrating the previous methods: a Rule as Feature model and a Classifier Confidence model. Several ML algorithms were tested, including logistic regression (LR), support vector machine (SVM), and eXtreme gradient boosting (XGB). RESULTS When training on data from a single institution, LR lags behind the rules by 3.5% (F1 score: 92.2% v 95.7%). Hybrid models, instead, obtain competitive results, with Classifier Confidence outperforming the rules by +0.5% (96.2%). When a larger amount of data from multiple institutions is used, LR improves by +1.5% over the rules (97.2%), whereas hybrid systems obtain +2.2% for Rule as Feature (97.7%) and +2.6% for Classifier Confidence (98.3%). Replacing LR with SVM or XGB yielded similar performance gains. CONCLUSION We developed methods to use pre-existing handcrafted rules to inform ML algorithms. These hybrid systems obtain better performance than either rules or ML models alone, even when training data are limited. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|