Predicting Vomiting in Oncology/HCT

• An EHR-based LightGBM model predicted any vomiting within 96 hours after the 08:30 prediction time; retrospective AUROC was 0.730 and prospective silent-trial AUROC was 0.716 (prospective n=340 admissions).

• Pharmacy impact is modest and actionable: expect ~15 alerts/week (≈12–13 oncology, ≈3–4 hematopoietic cell transplant [HCT]); oncology alert performance at the clinician-selected threshold was PPV 0.259 (NNA=3) and NPV 0.900, while HCT PPV was 0.512 (NNA=2) with NPV 0.842 — suitable for targeted pharmacist review accepting some false positives.

• Vomiting is common and under-controlled among pediatric oncology and HCT inpatients, worsening nutrition, prolonging care, and harming quality of life; early risk identification targets a frequent and clinically meaningful source of patient suffering.

• Pharmacy teams lack an efficient, standardized way to triage admissions for intensified antiemetic review; manual chart review is variable and time-consuming, limiting targeted pharmacist impact.

• A validated risk flag that integrates into morning admission review can focus pharmacist-led, guideline-consistent antiemetic optimization — but deployment must balance clinical decision support (CDS) alert burden, antiemetic stewardship, and finite pharmacy resources.

• Single-center study at SickKids of pediatric oncology and hematopoietic cell transplant (HCT) admissions: 7,408 retrospective admissions (2018–2024) and a 3-month prospective silent trial of 340 admissions; predictions were generated once per admission at 08:30 the morning following admission to forecast any vomiting within 96 hours.

• Built an EHR-based ML pipeline using the SEDAR curated extract; compared L2-regularized logistic regression, XGBoost and LightGBM, selecting LightGBM. Final model used ~2,859 features (demographics, meds, labs, diagnoses) and produced a daily high-risk flag intended for pharmacist review and antiemetic optimization.

• Validated with a temporal 70/15/15 train/validation/test split and a 3-month prospective silent trial; clinical pharmacist champions set operational thresholds (oncology NNA=3; HCT NNA=2). Practical note: features came from a midnight Epic Clarity snapshot and the model was blind to data recorded between 00:00 and 08:30; unit of analysis was admissions (repeat admissions allowed).

• Model discrimination: retrospective AUROC 0.730 (95% CI 0.694–0.765) and prospective silent-trial AUROC 0.716 (95% CI 0.649–0.784). Overall vomiting prevalence ≈25.2% (oncology ≈22.2%, HCT ≈39.3%).

• Clinician-selected thresholds produced prospective oncology PPV 0.259, NPV 0.900, sensitivity 0.774; prospective HCT PPV 0.512, NPV 0.842, sensitivity 0.880 — stronger rule-in performance for HCT, stronger rule-out utility in oncology.

• Expected alert burden: ≈12–13 oncology alerts/week and ≈3–4 HCT alerts/week (≈15 total/week), creating a manageable daily targeted-review workload for pharmacists given the chosen NNAs.

• The model retained performance prospectively using 2,859 EHR-derived features and operational thresholding informed by clinical champions, supporting feasibility for a pilot deployment contingent on local validation and monitoring.

• Operational drivers of the system’s utility were the high-dimensional EHR feature set plus clinician-selected thresholds that prioritized roughly 80% sensitivity.

• Implement a once-daily (08:30) High-Risk flag on the oncology/HCT pharmacist worklist derived from a midnight Clarity snapshot; assign pharmacist ownership for verification of pathway-consistent antiemetics and communication of specific recommendations to the medical team.

• Plan resourcing for ~15 alerts/week: designate morning coverage and adopt service-specific thresholds (Oncology NNA=3; HCT NNA=2). Use a brief checklist and micro-training to accelerate reviews (confirm emetogenicity, scheduled/PRN antiemetics, contraindications).

• Establish pharmacist-led governance to track alert rates, actions, vomiting outcomes, and fairness metrics; revalidate periodically and maintain an MLOps change-control process.

• Mitigate the 00:00–08:30 blind spot operationally by adding manual overnight checks for key events and by quantifying how frequently critical antiemetic actions occur in that window before rollout.

• Treat the tool as a morning ‘radar sweep’ — AI narrows where to look, and human clinical judgment remains the gate for any antiemetic changes.

Strengths:

• Prospective silent-trial validation preserved model performance (AUROC ~0.716), supporting temporal robustness beyond retrospective training.

• Operational focus with clinical pharmacist champions and a large, curated EHR feature set (2,859 features) enabled pragmatic thresholding, alert-volume planning, and a clear pilot pathway.

Limitations:

• Model is blind to clinical data recorded between midnight and the 08:30 prediction time, risking missed overnight events and misclassification unless mitigated operationally or by real-time data feeds.

• Inclusion of repeated admissions per patient and absence of explicit chemotherapy/regimen-level variables may inflate apparent performance and limit transportability to other sites.

A prospectively-validated LightGBM model demonstrates feasibility for pharmacist-led, guideline-focused antiemetic targeting in pediatric oncology/HCT, but safe deployment requires a local SEDAR-like data pipeline and MLOps/governance plus local silent-trial validation; expect >12 months to reach enterprise-ready pilot readiness without existing infrastructure.