Automated de-identification of free-text medical records

Autor: Gari D. Clifford, Margaret M Douglass, Ishna Neamatullah, George B. Moody, Mauricio Villarroel, Roger G. Mark, Andrew T. Reisner, Peter Szolovits, William J. Long, Li-wei H. Lehman
Přispěvatelé: Harvard University--MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science, Harvard--MIT Program in Health Sciences and Technology. Laboratory for Computational Physiology, Lehman, Li-Wei H., Reisner, Andrew T., Villarroel Montoya, Mauricio Christian, Long, William J., Szolovits, Peter, Moody, George B., Mark, Roger Greenwood, Clifford, Gari D., Neamatullah, Ishna, Douglass, Margaret M.
Rok vydání: 2008
Předmět:
020205 medical informatics
Computer science
Word count
Dictionaries as Topic
Health Informatics
02 engineering and technology
lcsh:Computer applications to medicine. Medical informatics
computer.software_genre
Health informatics
Medical Records
Set (abstract data type)
03 medical and health sciences
0302 clinical medicine
0202 electrical engineering
electronic engineering
information engineering
Humans
030212 general & internal medicine
Natural Language Processing
Protected health information
Health Insurance Portability and Accountability Act
business.industry
Health Policy
Medical record
De-identification
Patient Discharge
United States
3. Good health
Computer Science Applications
lcsh:R858-859.7
Programming Languages
Data mining
Artificial intelligence
Heuristics
business
computer
Algorithms
Confidentiality
Software
Natural language processing
Research Article
Zdroj: BMC Medical Informatics and Decision Making, Vol 8, Iss 1, p 32 (2008)
BioMed Central Ltd
BMC Medical Informatics and Decision Making
ISSN: 1472-6947
DOI: 10.1186/1472-6947-8-32
Popis: Background: Text-based patient medical records are a vital resource in medical research. In order to preserve patient confidentiality, however, the U.S. Health Insurance Portability and Accountability Act (HIPAA) requires that protected health information (PHI) be removed from medical records before they can be disseminated. Manual de-identification of large medical record databases is prohibitively expensive, time-consuming and prone to error, necessitating automatic methods for large-scale, automated de-identification. Methods: We describe an automated Perl-based de-identification software package that is generally usable on most free-text medical records, e.g., nursing notes, discharge summaries, X-ray reports, etc. The software uses lexical look-up tables, regular expressions, and simple heuristics to locate both HIPAA PHI, and an extended PHI set that includes doctors' names and years of dates. To develop the de-identification approach, we assembled a gold standard corpus of re-identified nursing notes with real PHI replaced by realistic surrogate information. This corpus consists of 2,434 nursing notes containing 334,000 words and a total of 1,779 instances of PHI taken from 163 randomly selected patient records. This gold standard corpus was used to refine the algorithm and measure its sensitivity. To test the algorithm on data not used in its development, we constructed a second test corpus of 1,836 nursing notes containing 296,400 words. The algorithm's false negative rate was evaluated using this test corpus. Results: Performance evaluation of the de-identification software on the development corpus yielded an overall recall of 0.967, precision value of 0.749, and fallout value of approximately 0.002. On the test corpus, a total of 90 instances of false negatives were found, or 27 per 100,000 word count, with an estimated recall of 0.943. Only one full date and one age over 89 were missed. No patient names were missed in either corpus. Conclusion We have developed a pattern-matching de-identification system based on dictionary look-ups, regular expressions, and heuristics. Evaluation based on two different sets of nursing notes collected from a U.S. hospital suggests that, in terms of recall, the software out-performs a single human de-identifier (0.81) and performs at least as well as a consensus of two human de-identifiers (0.94). The system is currently tuned to de-identify PHI in nursing notes and discharge summaries but is sufficiently generalized and can be customized to handle text files of any format. Although the accuracy of the algorithm is high, it is probably insufficient to be used to publicly disseminate medical data. The open-source de-identification software and the gold standard re-identified corpus of medical records have therefore been made available to researchers via the PhysioNet website to encourage improvements in the algorithm.
National Institute of Biomedical Imaging and Bioengineering (U.S.)
National Institutes of Health (U.S) ( Grant Number R01-EB001659 )
Databáze: OpenAIRE
Externí odkaz: