SHM Converge Scientific Abstract ePoster Gallery-Quality and Accuracy of Chatbot Generated Patient Discharge Instructions

Pdf Summary

This poster evaluates whether GPT-4 can generate patient-friendly discharge instructions from hospital discharge summaries. Using discharge notes from the MIMIC-IV database, the authors inputted the “Hospital Course” and “Discharge Medications” sections into ChatGPT via the Azure API after refining prompts over multiple iterations. Two hospital medicine physicians reviewed instructions from 15 cases and rated them for accuracy and quality on 1–5 scales. Readability was also assessed with the Flesch-Kincaid grade level.<br /><br />Results showed strong performance for accuracy, with a mean score of 4.57/5, while quality was somewhat lower at 3.8/5. The generated instructions were generally readable at about a 7th-grade level, close to the intended 6th-grade target. Reviewers noted that outputs were often personalized and clinically relevant, especially for medication changes and return precautions.<br /><br />However, the instructions also had recurring issues, including awkward wording, overly generic explanations, and occasionally stigmatizing language. The authors conclude that LLMs like GPT-4 show real promise for helping draft discharge instructions, but they still need careful physician review and proofreading to ensure completeness, clarity, and professionalism.

Asset Subtitle

Peter Barish

Meta Tag

Author List Karl Swanson, Peter Barish, Sara Murray, Smitha Ganeshan

Category Research

Concept Large Language Model

Concept Patient Discharge Instruction

Concept ChatGPT 4

Concept Prompt Engineering

Concept Clinical Documentation

Distinguished Non-Finalist

Presenter Organization University of California, San Francisco

Presenting Author Peter Barish

Track Technology in Hospital Medicine

Keywords

GPT-4

discharge instructions

hospital discharge summaries

MIMIC-IV

ChatGPT

Azure API

readability

Flesch-Kincaid grade level

patient-friendly

physician review

Large Language Model

Patient Discharge Instruction

ChatGPT 4

Prompt Engineering

Clinical Documentation