Comparative performance of intensive care mortality prediction models based on manually curated versus automatically extracted electronic health record data.

Autor: Jagesar AR; Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam Public Health (APH), Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands; Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, the Netherlands. Electronic address: a.jagesar@amsterdamumc.nl., Otten M; Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam Public Health (APH), Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands; Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, the Netherlands., Dam TA; Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam Public Health (APH), Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands; Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, the Netherlands., Biesheuvel LA; Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam Public Health (APH), Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands; Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, the Netherlands., Dongelmans DA; Department of Intensive Care Medicine, Amsterdam UMC, Universiteit van Amsterdam, Amsterdam, the Netherlands., Brinkman S; Department of Medical Informatics, Amsterdam UMC, Amsterdam Public Health Research Institute and National Intensive Care Evaluation (NICE) Foundation, Amsterdam, the Netherlands., Thoral PJ; Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam Public Health (APH), Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands., François-Lavet V; Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, the Netherlands., Girbes ARJ; Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam Public Health (APH), Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands., de Keizer NF; Department of Medical Informatics, Amsterdam UMC, Amsterdam Public Health Research Institute and National Intensive Care Evaluation (NICE) Foundation, Amsterdam, the Netherlands., de Grooth HJS; Intensive Care Center, UMC Utrecht, Utrecht, The Netherlands., Elbers PWG; Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam Public Health (APH), Amsterdam UMC, Vrije Universiteit, Amsterdam, the Netherlands.
Jazyk: angličtina
Zdroj: International journal of medical informatics [Int J Med Inform] 2024 Aug; Vol. 188, pp. 105477. Date of Electronic Publication: 2024 May 08.
DOI: 10.1016/j.ijmedinf.2024.105477
Abstrakt: Introduction: Benchmarking intensive care units for audit and feedback is frequently based on comparing actual mortality versus predicted mortality. Traditionally, mortality prediction models rely on a limited number of input variables and significant manual data entry and curation. Using automatically extracted electronic health record data may be a promising alternative. However, adequate data on comparative performance between these approaches is currently lacking.
Methods: The AmsterdamUMCdb intensive care database was used to construct a baseline APACHE IV in-hospital mortality model based on data typically available through manual data curation. Subsequently, new in-hospital mortality models were systematically developed and evaluated. New models differed with respect to the extent of automatic variable extraction, classification method, recalibration usage and the size of collection window.
Results: A total of 13 models were developed based on data from 5,077 admissions divided into a train (80%) and test (20%) cohort. Adding variables or extending collection windows only marginally improved discrimination and calibration. An XGBoost model using only automatically extracted variables, and therefore no acute or chronic diagnoses, was the best performing automated model with an AUC of 0.89 and a Brier score of 0.10.
Discussion: Performance of intensive care mortality prediction models based on manually curated versus automatically extracted electronic health record data is similar. Importantly, our results suggest that variables typically requiring manual curation, such as diagnosis at admission and comorbidities, may not be necessary for accurate mortality prediction. These proof-of-concept results require replication using multi-centre data.
Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
(Copyright © 2024. Published by Elsevier B.V.)
Databáze: MEDLINE
Externí odkaz: