Validation of a Natural Language Processing Algorithm for Detecting Infectious Disease Symptoms in Primary Care Electronic Medical Records in Singapore
Autor: | Win Wah, Mong Li Lee, Wynne Hsu, Franco Wong, Gilbert Lim, Sok Huang Teo, Mohammed Ridzwan Bin Abdullah, Mark I-Cheng Chen, Vernon J. Lee, Antony Hardjojo, Long Pang, Arunan Gunachandran, Jonathan Siung King Phang, Joash Wen Chen Chong, Ee Hui Goh |
---|---|
Rok vydání: | 2018 |
Předmět: |
medicine.medical_specialty
020205 medical informatics education communicable diseases Health Informatics 02 engineering and technology computer.software_genre 03 medical and health sciences 0302 clinical medicine Health Information Management Health care Epidemiology 0202 electrical engineering electronic engineering information engineering Sore throat medicine syndromic surveillance 030212 general & internal medicine natural language processing Original Paper Disease surveillance Recall business.industry Medical record electronic health records Infectious disease (medical specialty) surveillance epidemiology Artificial intelligence medicine.symptom business Precision and recall Algorithm computer Natural language processing |
Zdroj: | JMIR Medical Informatics |
ISSN: | 2291-9694 |
Popis: | Background: Free-text clinical records provide a source of information that complements traditional disease surveillance. To electronically harness these records, they need to be transformed into codified fields by natural language processing algorithms. Objective: The aim of this study was to develop, train, and validate Clinical History Extractor for Syndromic Surveillance (CHESS), an natural language processing algorithm to extract clinical information from free-text primary care records. Methods: CHESS is a keyword-based natural language processing algorithm to extract 48 signs and symptoms suggesting respiratory infections, gastrointestinal infections, constitutional, as well as other signs and symptoms potentially associated with infectious diseases. The algorithm also captured the assertion status (affirmed, negated, or suspected) and symptom duration. Electronic medical records from the National Healthcare Group Polyclinics, a major public sector primary care provider in Singapore, were randomly extracted and manually reviewed by 2 human reviewers, with a third reviewer as the adjudicator. The algorithm was evaluated based on 1680 notes against the human-coded result as the reference standard, with half of the data used for training and the other half for validation. Results: The symptoms most commonly present within the 1680 clinical records at the episode level were those typically present in respiratory infections such as cough (744/7703, 9.66%), sore throat (591/7703, 7.67%), rhinorrhea (552/7703, 7.17%), and fever (928/7703, 12.04%). At the episode level, CHESS had an overall performance of 96.7% precision and 97.6% recall on the training dataset and 96.0% precision and 93.1% recall on the validation dataset. Symptoms suggesting respiratory and gastrointestinal infections were all detected with more than 90% precision and recall. CHESS correctly assigned the assertion status in 97.3%, 97.9%, and 89.8% of affirmed, negated, and suspected signs and symptoms, respectively (97.6% overall accuracy). Symptom episode duration was correctly identified in 81.2% of records with known duration status. Conclusions: We have developed an natural language processing algorithm dubbed CHESS that achieves good performance in extracting signs and symptoms from primary care free-text clinical records. In addition to the presence of symptoms, our algorithm can also accurately distinguish affirmed, negated, and suspected assertion statuses and extract symptom durations. |
Databáze: | OpenAIRE |
Externí odkaz: |