Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare.

Autor: Guo LL; Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada., Morse KE; Division of Pediatric Hospital Medicine, Department of Pediatrics, Stanford University, Palo Alto, CA, USA., Aftandilian C; Division of Hematology/Oncology, Department of Pediatrics, Stanford University, Palo Alto, CA, USA., Steinberg E; Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA., Fries J; Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA., Posada J; Universidad del Norte, Barranquilla, Colombia., Fleming SL; Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA., Lemmon J; Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada., Jessa K; Information Services, The Hospital for Sick Children, Toronto, ON, Canada., Shah N; Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA., Sung L; Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada. Lillian.sung@sickkids.ca.; Division of Haematology/Oncology, The Hospital for Sick Children, 555 University Avenue, M5G1X8, Toronto, ON, Canada. Lillian.sung@sickkids.ca.
Jazyk: angličtina
Zdroj: BMC medical informatics and decision making [BMC Med Inform Decis Mak] 2024 Feb 14; Vol. 24 (1), pp. 51. Date of Electronic Publication: 2024 Feb 14.
DOI: 10.1186/s12911-024-02449-8
Abstrakt: Background: Diagnostic codes are commonly used as inputs for clinical prediction models, to create labels for prediction tasks, and to identify cohorts for multicenter network studies. However, the coverage rates of diagnostic codes and their variability across institutions are underexplored. The primary objective was to describe lab- and diagnosis-based labels for 7 selected outcomes at three institutions. Secondary objectives were to describe agreement, sensitivity, and specificity of diagnosis-based labels against lab-based labels.
Methods: This study included three cohorts: SickKids from The Hospital for Sick Children, and StanfordPeds and StanfordAdults from Stanford Medicine. We included seven clinical outcomes with lab-based definitions: acute kidney injury, hyperkalemia, hypoglycemia, hyponatremia, anemia, neutropenia and thrombocytopenia. For each outcome, we created four lab-based labels (abnormal, mild, moderate and severe) based on test result and one diagnosis-based label. Proportion of admissions with a positive label were presented for each outcome stratified by cohort. Using lab-based labels as the gold standard, agreement using Cohen's Kappa, sensitivity and specificity were calculated for each lab-based severity level.
Results: The number of admissions included were: SickKids (n = 59,298), StanfordPeds (n = 24,639) and StanfordAdults (n = 159,985). The proportion of admissions with a positive diagnosis-based label was significantly higher for StanfordPeds compared to SickKids across all outcomes, with odds ratio (99.9% confidence interval) for abnormal diagnosis-based label ranging from 2.2 (1.7-2.7) for neutropenia to 18.4 (10.1-33.4) for hyperkalemia. Lab-based labels were more similar by institution. When using lab-based labels as the gold standard, Cohen's Kappa and sensitivity were lower at SickKids for all severity levels compared to StanfordPeds.
Conclusions: Across multiple outcomes, diagnosis codes were consistently different between the two pediatric institutions. This difference was not explained by differences in test results. These results may have implications for machine learning model development and deployment.
(© 2024. The Author(s).)
Databáze: MEDLINE
Externí odkaz:
Nepřihlášeným uživatelům se plný text nezobrazuje