EHR Triage Flags High-Yield Antibiotic Orders

• An interpretable XGBoost model using routinely available electronic health record (EHR

• Epic/Clarity) vitals, laboratory, and microbiology data flagged 425 antibiotic orders with area under the receiver operating characteristic curve (AUROC) 0.89 for necessity and 0.80 for optimal selection.

• The model can help antibiotic stewardship programs (ASPs) and pharmacists prioritize high-yield prospective audit-and-feedback (PAF) reviews, reducing manual workload and focusing interventions on unstable or inflammatory cases.

• Hospitalized patients frequently receive antibiotics, and a sizable fraction of in-hospital use is unnecessary or suboptimal, contributing to adverse drug events, Clostridioides difficile infection, and antimicrobial resistance while requiring empiric decisions with limited early data.

• Prospective audit-and-feedback (PAF) improves appropriateness but is labor-intensive and cannot feasibly cover the volume of orders

• many hospitals therefore perform limited, periodic point-prevalence reviews that leave many potentially inappropriate orders unreviewed.

• Automated, EHR-based prescreening to triage antibiotic orders could enable stewardship teams and pharmacists to concentrate scarce resources on unstable or high-yield cases and inform clinical decision support and operational resource allocation.

• Single-center study at a tertiary academic hospital reviewing inpatient antibiotic orders on hospital medicine services between May 2021 and August 2022

• intensive care, surgical, cardiology, malignant hematology services, and patients with infectious diseases consultation were excluded.

• Three infectious-disease experts performed point-prevalence reviews using the National Antimicrobial Prescribing Survey (NAPS) to label individual orders as 'indicated' or 'optimal' for use.

• EHR data were extracted from Epic/Clarity (vitals, laboratories, microbiology, medication administration record), censored to the day before review, and used to train an interpretable time-series XGBoost model with 10-fold cross-validation

• performance was compared to simpler clinical scores to prioritize orders for stewardship review.

• XGBoost (n = 425 orders) achieved AUROC 0.89 (95% CI 0.87–0.91) for necessity and 0.80 (95% CI 0.77–0.83) for optimal antimicrobial selection

• reviewers labeled 75% of orders as 'indicated' and 56% as 'optimal'.

• Models using only vital signs performed strongly (AUROC 0.87/0.77 indicated/optimal)

• laboratory-only models 0.76/0.76

• microbiology/medication administration record (MAR) alone performed worse (0.74/0.66)

• systemic inflammatory response syndrome (SIRS) criteria performed poorly (AUROC 0.42).

• Performance was consistent across antibiotic spectra: broad-spectrum hospital onset (BSHO) 0.86/0.77, broad-spectrum community onset (BSCA) 0.87/0.81, MRSA 0.87/0.77, ESBL 0.89/0.73, Pseudomonas 0.82/0.78.

• SHAP (SHapley Additive exPlanations) identified hemodynamic variability, inflammatory markers, and recent laboratory trends as the dominant drivers of predictions rather than microbiology or antibiotic class.

• Practically, automated triage could prioritize high-yield PAF early in therapy (60% of orders < 48 hours), concentrating review on unstable or inflammatory cases driven mainly by vital sign and lab trends rather than early microbiology results.

• Integrate the model into a daily Epic/Clarity feed and present prioritized antibiotic orders on a stewardship dashboard or workqueue to focus prospective audit-and-feedback on unstable/inflammatory cases and early therapy orders.

• Operationalize via a nightly Clarity export, calibrate alert thresholds to match ASP and pharmacy capacity, and display SHAP-derived feature drivers in the user interface so pharmacists can rapidly interpret why an order was flagged.

• Treat the tool as an AI 'triage nurse'—it sorts high-yield orders for human review, enabling stewardship teams to allocate time to complex cases while preserving final clinical judgment.

• Maintain pharmacist oversight and governance: require pharmacist sign-off on flagged orders, exclude infectious diseases–consulted patients per model training, and perform periodic audits to monitor performance and guide refinements.

Strengths:

• Uses an interpretable time-series XGBoost model with SHAP and reporting aligned to TRIPOD guidelines, enhancing transparency and reproducibility.

• Relies on routinely available structured EHR data (vitals, labs) with automated extraction, supporting scalable, timely triage without manual chart review.

Limitations:

• Single-center, hospital-medicine–only cohort and modest sample size limit external validity across intensive care units, surgical services, and community hospitals.

• Model excludes unstructured clinical notes and patients with infectious diseases consultation

• periodic NAPS point-prevalence labeling with inter-rater variability may miss contextual factors (for example, prehospital antibiotics or nuanced comorbidities) that affect label fidelity and deployment performance.

This interpretable XGBoost triage model reliably prioritizes high-yield antibiotic orders and is ready for real-world pilot deployment within ASP and pharmacy workflows with local calibration and continued pharmacist governance.