SHM Converge Scientific Abstract ePoster Gallery-Transforming Discharge Summaries Into Patient-Friendly Language & Format Using GPT-4

Pdf Summary

This cross-sectional study examined whether a large language model (LLM), using Microsoft Azure OpenAI/GPT-4, could convert inpatient discharge summaries into patient-friendly language and format. The researchers analyzed discharge summaries from 50 adult patients discharged from NYU Langone Health’s General Internal Medicine service in June 2023.<br /><br />The transformed summaries were much shorter and easier to read than the original electronic health record versions. Mean word count dropped from 1,520 to 338 words. The Flesch-Kincaid Grade Level improved from 11.0 to 6.2, indicating a lower reading level. PEMAT understandability scores also improved substantially, from 13% to 81%.<br /><br />To assess quality, two physicians independently reviewed each patient-friendly summary for accuracy and completeness. Of 100 total reviews, 54% gave the highest accuracy rating, and 56% rated the summaries as completely complete. However, 18 reviews identified safety concerns, mostly due to omitted information, though some inaccuracies or “hallucinations” were also noted.<br /><br />Overall, the study suggests that LLMs can effectively translate discharge summaries into more readable, understandable patient-facing documents. Still, the findings show that current systems are not sufficiently reliable for unsupervised use. The authors conclude that physician review will be necessary at first, and that further improvements are needed in accuracy, completeness, and safety before broader implementation.

Asset Subtitle

Jonah Zaretsky

Meta Tag

Author List Jeong Min Kim, Jonah Feldman, Jonah Zaretsky, Jonathan Austrian, Ravi Gupta, Samuel Baskharoun, Saul Blecker, Yindalon Aphinyanaphongs, Yunan Zhao

Category Research

Concept Large Language Model

Concept Inpatient Discharge Summary

Concept Hallucination

Concept Physician Review

Concept Accessible Language

Distinguished Non-Finalist

Presenter Organization NYU Langone Health

Presenting Author Jonah Zaretsky

Track Communication

Keywords

large language model

GPT-4

discharge summaries

patient-friendly language

readability

Flesch-Kincaid Grade Level

PEMAT understandability

accuracy review

safety concerns

healthcare documentation

Large Language Model

Inpatient Discharge Summary

Hallucination

Physician Review

Accessible Language