Comparison of NLP machine learning models with human physicians for ASA Physical Status classification

Autor: Soo Bin Yoon, Jipyeong Lee, Hyung-Chul Lee, Chul-Woo Jung, Hyeonhoon Lee
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: npj Digital Medicine, Vol 7, Iss 1, Pp 1-11 (2024)
Druh dokumentu: article
ISSN: 2398-6352
41458591
DOI: 10.1038/s41746-024-01259-6
Popis: Abstract The American Society of Anesthesiologist’s Physical Status (ASA-PS) classification system assesses comorbidities before sedation and analgesia, but inconsistencies among raters have hindered its objective use. This study aimed to develop natural language processing (NLP) models to classify ASA-PS using pre-anesthesia evaluation summaries, comparing their performance to human physicians. Data from 717,389 surgical cases in a tertiary hospital (October 2004–May 2023) was split into training, tuning, and test datasets. Board-certified anesthesiologists created reference labels for tuning and test datasets. The NLP models, including ClinicalBigBird, BioClinicalBERT, and Generative Pretrained Transformer 4, were validated against anesthesiologists. The ClinicalBigBird model achieved an area under the receiver operating characteristic curve of 0.915. It outperformed board-certified anesthesiologists with a specificity of 0.901 vs. 0.897, precision of 0.732 vs. 0.715, and F1-score of 0.716 vs. 0.713 (all p
Databáze: Directory of Open Access Journals
Externí odkaz: