SHM Converge Scientific Abstract ePoster Gallery-How Patient-Centered Are Advance Care Planning Notes? A Scalable, Clinician-Validated LLM Comparison of Structured Template vs. Narrative Acp Documentation

How Patient-Centered Are Advance Care Planning Notes? A Scalable, Clinician-Validated LLM Comparison of Structured Template vs. Narrative Acp Documentation

Back to course

Pdf Summary

This study examined whether structured advance care planning (ACP) templates produce more patient-centered documentation than narrative ACP notes. Because ACP documentation in electronic health records often focuses on treatment decisions rather than patient values, the authors developed a clinician-validated large language model (LLM) pipeline to measure patient-centeredness.<br /><br />They created a 9-item binary rubric, adapted from the ACP Communication Assessment Tool, covering topics such as understanding prognosis, goals, fears, valued activities, healthcare proxy, and care recommendations. Three clinicians iteratively refined the rubric until agreement stabilized.<br /><br />Using a retrospective chart review of hospitalized patients at a tertiary academic center, the team analyzed ACP notes from 2011–2025. After exclusions, the dataset included 483 pre-intervention notes and 488 post-intervention notes. The structured ACP template was introduced in September 2018.<br /><br />Validation showed high clinician inter-rater agreement across rubric items (mean pairwise agreement 0.83–0.96) and 94% accuracy against consensus labels. The Qwen-based LLM achieved 80% agreement with clinician consensus, with the biggest mismatches in identifying fears/worries, critical functions for quality of life, and valued life activities.<br /><br />Results showed that structured template-based ACP notes were more patient-centered than narrative notes. Mean patient-centeredness scores increased after template implementation (5.28 vs. 5.05, p=0.04). Within the post-intervention period, structured notes scored substantially higher than narrative notes (6.42 vs. 5.05, p=0.001).<br /><br />Overall, the study concludes that structured ACP documentation improves patient-centeredness and that the clinician-validated LLM framework offers a scalable way to evaluate ACP notes and guide documentation improvements.

Asset Subtitle

Samantha Wang

Meta Tag

Author List Jiyeong Kim, Michelle Low, Samantha Wang, Stephen P. Ma, Winifred Teuteberg

Category Research

Concept Advance care planning

Concept Patient-Centered Care

Concept Clinician-Validated Rubric

Concept Retrospective Cohort Study

Concept Large Language Model

Distinguished Non-Finalist

Presenter Organization Stanford University

Presenting Author Samantha Wang

Track Technology in Hospital Medicine

Keywords

advance care planning

patient-centered documentation

structured templates

narrative notes

large language model

electronic health records

prognosis

healthcare proxy

retrospective chart review

clinician validation

Advance care planning

Patient-Centered Care

Clinician-Validated Rubric

Retrospective Cohort Study

Large Language Model