Enhancing Health Literacy: Evaluating the Readability of Patient Handouts Revised by ChatGPT's Large Language Model.

Autor: Swisher AR; Department of Otolaryngology-Head and Neck Surgery, Mayo Clinic, Phoenix, Arizona, USA., Wu AW; Division of Otolaryngology-Head and Neck Surgery, Cedars-Sinai, Los Angeles, California, USA., Liu GC; Division of Otolaryngology-Head and Neck Surgery, Cedars-Sinai, Los Angeles, California, USA., Lee MK; Division of Otolaryngology-Head and Neck Surgery, Cedars-Sinai, Los Angeles, California, USA., Carle TR; Division of Otolaryngology-Head and Neck Surgery, Cedars-Sinai, Los Angeles, California, USA., Tang DM; Division of Otolaryngology-Head and Neck Surgery, Cedars-Sinai, Los Angeles, California, USA.
Jazyk: angličtina
Zdroj: Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery [Otolaryngol Head Neck Surg] 2024 Dec; Vol. 171 (6), pp. 1751-1757. Date of Electronic Publication: 2024 Aug 06.
DOI: 10.1002/ohn.927
Abstrakt: Objective: To use an artificial intelligence (AI)-powered large language model (LLM) to improve readability of patient handouts.
Study Design: Review of online material modified by AI.
Setting: Academic center.
Methods: Five handout materials obtained from the American Rhinologic Society (ARS) and the American Academy of Facial Plastic and Reconstructive Surgery websites were assessed using validated readability metrics. The handouts were inputted into OpenAI's ChatGPT-4 after prompting: "Rewrite the following at a 6th-grade reading level." The understandability and actionability of both native and LLM-revised versions were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Results were compared using Wilcoxon rank-sum tests.
Results: The mean readability scores of the standard (ARS, American Academy of Facial Plastic and Reconstructive Surgery) materials corresponded to "difficult," with reading categories ranging between high school and university grade levels. Conversely, the LLM-revised handouts had an average seventh-grade reading level. LLM-revised handouts had better readability in nearly all metrics tested: Flesch-Kincaid Reading Ease (70.8 vs 43.9; P < .05), Gunning Fog Score (10.2 vs 14.42; P < .05), Simple Measure of Gobbledygook (9.9 vs 13.1; P < .05), Coleman-Liau (8.8 vs 12.6; P < .05), and Automated Readability Index (8.2 vs 10.7; P = .06). PEMAT scores were significantly higher in the LLM-revised handouts for understandability (91 vs 74%; P < .05) with similar actionability (42 vs 34%; P = .15) when compared to the standard materials.
Conclusion: Patient-facing handouts can be augmented by ChatGPT with simple prompting to tailor information with improved readability. This study demonstrates the utility of LLMs to aid in rewriting patient handouts and may serve as a tool to help optimize education materials.
Level of Evidence: Level VI.
(© 2024 American Academy of Otolaryngology–Head and Neck Surgery Foundation.)
Databáze: MEDLINE
Externí odkaz: