SHM Converge Scientific Abstract ePoster Gallery-Real-Time Performance of a Machine-Learning Model for Admission Prediction

Pdf Summary

This report describes the real-time clinical deployment of a machine-learning model designed to predict whether emergency department (ED) patients will require hospital admission. The model was developed by Mayo Clinic internal medicine and emergency medicine teams to support triage during COVID-related crowding by helping identify patients who clearly needed admission and those who could likely be treated safely as outpatients.<br /><br />The model uses a gradient-boosted trees algorithm and 46 features available early in the ED course from the electronic health record (EHR). It was trained on Mayo Clinic data from 2018–2019 using a 70/15/15 train/validation/test split. The model was then integrated into Mayo’s Epic EHR Cognitive Compute platform, with the admission risk score displayed directly on the ED patient trackboard for real-time use.<br /><br />In live clinical use at the Rochester ED, the model achieved an area under the receiver-operating characteristic curve (AUC) of 0.81, compared with 0.84 during development. Calibration was also strong, with a Brier score of 0.159 and log loss of 0.485, indicating that predicted probabilities reasonably matched actual admission rates.<br /><br />The authors note that performance remained robust despite technical challenges related to feature selection and migration into the live EHR environment. The model is now being considered for use at additional sites within the health system, and a clinical trial is underway to measure its effect on triage efficiency and hospital flow.<br /><br />Overall, the project shows that accurate, real-time admission prediction using routinely available EHR data is feasible and can potentially improve ED triage and disposition decisions.

Asset Subtitle

Ray Qian

Meta Tag

Author List Alexander J. Ryu, Ray Qian, Sean Legler, Shant Ayanian

Category Innovations

Concept Boosted Tree Ensemble

Concept Emergency Department Admission

Concept Electronic Health Record

Concept Train-Test Split

Concept AUC

Distinguished Non-Finalist

Presenter Organization Mayo Clinic Division of Hospital Internal Medicine

Presenting Author Ray Qian

Track Technology in Hospital Medicine

Keywords

machine learning

emergency department

hospital admission prediction

clinical deployment

EHR data

triage support

gradient-boosted trees

real-time risk score

Epic EHR

COVID crowding

Boosted Tree Ensemble

Emergency Department Admission

Electronic Health Record

Train-Test Split

AUC