SHM Converge Scientific Abstract ePoster Gallery-Analysis of Clinical Questions Asked and Answered of a Rag-Enhanced Ai Chatbot, Vlrchat

Pdf Summary

This study analyzed authentic clinical questions asked to VLRChat, a retrieval-augmented AI chatbot built on the Vanderbilt Internal Medicine Housestaff Handbook (VIMbook.org). The goal was to identify educational gaps and user needs by examining what clinicians actually ask at the point of care.<br /><br />Researchers retrospectively reviewed de-identified question-answer logs from February to July 2025. From 1,000 collected questions, 250 were randomly sampled; 33 were excluded, leaving 217 usable questions for analysis.<br /><br />Key findings showed that treatment questions were most common (49%), reflecting the need for real-time bedside decision support. Questions most often involved cardiology, infectious disease, and gastroenterology, suggesting possible educational gaps in these specialties. Forty-two questions were asked after hours (7 PM–7 AM), indicating that VLRChat may serve as an independent learning resource outside normal clinical workflow. Smaller but notable categories included 8 education-related questions and 5 patient communication questions, showing that the chatbot may also support learning and communication, not just direct treatment decisions.<br /><br />The authors conclude that analyzing question patterns can help reveal unmet learning needs and inform curriculum development. They argue that AI tools like VLRChat can promote authentic clinical question asking while providing accurate, institution-specific answers through retrieval-augmented generation.

Asset Subtitle

Ivo Su

Meta Tag

Author List C. Beau Hilton, Chase Jeffrey J. Webber DO FACP, Ivo Su, Joshua Onyango MD MEd, Matthew Sweeney, Michael Neuss, Sameer Sundrani, Shane Stenner, Snehal Bindra

Category Innovations

Concept VLRChat

Concept Retrieval-Augmented Generation

Concept Clinical Query

Concept Vanderbilt Internal Medicine Housestaff Handbook

Concept Treatment

Distinguished Non-Finalist

Presenter Organization Vanderbilt University School of Medicine

Presenting Author Ivo Su

Track Technology in Hospital Medicine

Keywords

retrieval-augmented generation

clinical questions

point-of-care decision support

VLRChat

VIMbook

treatment questions

educational gaps

cardiology

infectious disease

gastroenterology

VLRChat

Retrieval-Augmented Generation

Clinical Query

Vanderbilt Internal Medicine Housestaff Handbook

Treatment