Pharmacists Use EHR Risk Flags to Prioritize Paxlovid

• An electronic health record (EHR)-based prediction model using 19 implementable risk factors achieved an area under the curve (AUC) of 0.825 for 14-day hospitalization or death; in the full sample the highest-risk 10% captured 48% of events with a positive predictive value (PPV) of 9.9%.

• Model implemented at Kaiser Permanente Washington (KPWA) with a 90th-percentile threshold for patients aged 18–64 enables pharmacists to prioritize Paxlovid (nirmatrelvir/ritonavir) counseling and drug–drug interaction management for flagged patients and to support more equitable outreach.

• COVID-19 continues to cause serious outcomes; outpatient antivirals can reduce hospitalization and death but identifying who benefits is difficult because risk factors are numerous, evidence is mixed, and therapy must start within five days of symptom onset.

• For pharmacy operations, treating broadly is not feasible: a Paxlovid course carries substantial cost (≈$1,500 average wholesale price), has common side effects, and creates clinically significant drug–drug interactions that require pharmacist assessment and targeted counseling.

• Health equity and capacity constraints persist—non-Hispanic Black and Latino patients are less likely to receive Paxlovid—and thresholds must track changing absolute risk and available staff. EHR-integrated risk guidance supports stewardship-oriented clinical decision support (CDS) under resource limits.

• Assembled 67,530 outpatient-documented COVID-19 infections (April 1, 2020–November 1, 2022) from KPWA EHR and claims for adults ≥18, excluding infections first documented on admission and any infections treated with outpatient antivirals.

• Performed variable selection from >100 candidate predictors in a research data warehouse to identify 19 EHR-implementable predictors, then refit a penalized logistic regression model (ridge) using EHR-derived predictors for real-time scoring after initial LASSO selection.

• Validated the model with ten-fold cross-validation, bootstrap confidence intervals, subgroup checks by age and race/ethnicity, iterative two-month forward (prospective) validation, and tested percentile-based thresholds in 18–64-year-olds while retaining a composite race/ethnicity indicator to support equitable EHR flags.

• Cross-validated AUC 0.825 (95% CI 0.813–0.836) overall and 0.802 (0.787–0.818) in 18–64-year-olds. Composite 14-day event rate was 2.0% (1,378/67,530).

• Flagging the top 10% of predicted risk yielded a true positive rate (TPR) of 48% (46–51%) overall with PPV 9.9% (9.2–10.6%); in 18–64-year-olds TPR was 47% with PPV 7.1%.

• Prospective validation showed discrimination remained acceptable but classification metrics varied over time: PPV declined from 15–22% (Oct 2020–mid‑2021) to <7% in the last six months, and fixed 90th‑percentile thresholds often flagged fewer than the expected 10% of infections.

• In the 18–64 non-immunocompromised subgroup (n=18,717; event rate 4.8/1,000), moving the cutoff from the 95th to the 90th percentile raised TPR from ~15% to ~27%; PPV exceeded 1% at ≥85th percentile and negative predictive value (NPV) remained >0.98. Major drivers of risk concentration were age, vaccination history, and comorbidity burden.

• Create an EHR workqueue for COVID‑positive patients and start with a 90th‑percentile cutoff for ages 18–64 without immunocompromising conditions or chronic lung disease; surface the model drivers (age, vaccination, comorbidity) so pharmacists can focus Paxlovid counseling and interaction checks within the five‑day window.

• Govern thresholds, not just the model: monitor a dashboard of PPV, TPR, and percent flagged monthly and adjust thresholds (e.g., between the 85th–95th percentiles) as capacity, variant severity, and absolute risk shift; recalibrate model predictions when base rates drift.

• Use the model’s composite race/ethnicity indicator to support equitable outreach—monitor flags, outreach attempts, and treatment acceptance by race/ethnicity and course-correct if disparities emerge.

• Operational insight: treat the score as radar, not autopilot—the model identifies elevated risk early, while pharmacists apply clinical judgment, document overrides, and ensure counseling and interaction management occur within the five‑day treatment window.

Strengths:

• Rigorous development pipeline: LASSO-based variable selection followed by an EHR-fitted ridge-penalized logistic regression with ten-fold cross-validation, bootstrap confidence intervals, and subgroup checks.

• Practice-ready validation: prospective forward validation, percentile-threshold testing for operational use, and an explicit equity assessment via a composite race/ethnicity indicator to support targeted outreach.

Limitations:

• Outcome was 14‑day all‑cause hospitalization or death; inability to isolate COVID‑attributable events may affect calibration and PPV for COVID-specific benefit estimates.

• Single health system sample limited to outpatient-documented infections without outpatient antiviral treatment; changing testing behavior and population risk over time caused threshold drift that requires local recalibration and ongoing monitoring.

A validated, EHR-deployable risk model (19 predictors) can drive pharmacy-led, equity‑oriented Paxlovid outreach and drug–drug interaction management, provided organizations actively tune thresholds and recalibrate predictions as risk and resources change.