SHM Converge Scientific Abstract ePoster Gallery-Making Written Feedback Comments Meaningful

Pdf Summary

The document describes a quality improvement project aimed at making narrative feedback from hospitalists to third-year medical students more meaningful.<br /><br />The team first created a 3-point rubric to score evaluation comments in three domains: positive feedback, constructive feedback, and actions/improvement plans. Each category was scored based on whether comments were absent, present but nonspecific, or specific to a skill or behavior. They then tested the rubric’s reliability using 146 evaluations from 2018–2019 scored by three blinded reviewers, finding high interrater reliability (intraclass correlation coefficient 0.959).<br /><br />Using this rubric, they scored baseline feedback and then implemented faculty development in stages. Initial intervention included general faculty development, followed by personalized score reports for faculty. These reports included a nomogram showing each faculty member’s score compared with peers. The project then expanded to division meetings and one-on-one meetings with every faculty member from January to March 2021. The feedback process emphasized two key themes: reinforcing effective feedback and improving actionable constructive feedback.<br /><br />Results showed baseline average narrative feedback scores of 3.6/6, with positive comments scoring highest (1.8/2) and constructive comments and improvement plans both scoring 0.9/2. A later example for one faculty member showed improvement, with a total score of 4.0/6. The summary also reports follow-up scoring of narrative feedback from 2021–2022 among the same hospitalists, suggesting the intervention aimed to track changes over time and improve the quality of written feedback.

Asset Subtitle

Alan M. Hall

Meta Tag

Author List Adam Gray, Alan M. Hall, John W. Ragsdale

Category Innovations

Concept Scoring Rubric

Concept Narrative Feedback

Concept Reliability

Concept Narrative Feedback Quality

Concept Action plan

Distinguished Non-Finalist

Presenter Organization University of Kentucky College of Medicine

Presenting Author Alan M. Hall

Track Education

Keywords

quality improvement

narrative feedback

hospitalists

medical students

evaluation rubric

interrater reliability

faculty development

constructive feedback

action plans

feedback quality

Scoring Rubric

Narrative Feedback

Reliability

Narrative Feedback Quality

Action plan