Triage-First AI Cuts Review Time for Cardiac Device Infections

Generative Model DevelopmentGenerative Process Outcomes

Authors:Dipandita Basnet; Hillary J Mull; Daniel J Morgan, et al.

Journal:Infection Control Hospital Epidemiology

DOI:10.1017/ice.2025.10384

AI-powered surveillance transforms cardiac device infection reviews from 25-minute chart reads into 5-7 minute verification tasks. Perfect for busy pharmacy teams who need to catch infections quickly without sacrificing accuracy or oversight.

January 23, 2026

Quick Take

An ML trigger flagged 334 of 12,927 CIED procedures (2.58%); in a stratified, retrospective 100-case test sample the VA GPT (GPT‑4o, temperature=0) identified all 50 sampled infections and correctly ruled out the 50 sampled non-infections (100% sensitivity and 100% specificity for infection detection in this enriched sample). For classifying whether an infection was procedure-related (correct timing/attribution) reported performance was lower: sensitivity ≈82% (95% CI 68.6%–91.4%) with 100% specificity. The GenAI-assisted workflow reduced per-chart review time from ~25 minutes to ~5–7 minutes in this study (timings included manual note extraction/copy‑paste as performed).

Operational model: use a less‑computational ML pre‑filter to triage charts, apply an LLM to summarize and extract structured elements, and have pharmacists/infection‑preventionists verify timelines and attributions — shifting work from exhaustive note reading to concise verification while retaining human oversight.

Why it Matters

Device-related healthcare‑associated infections are rare and often buried in long, unstructured notes; manual chart review is time‑consuming and not scalable.

Deciding whether a CIED infection is procedure‑related requires reconstructing timelines (90‑day window) from scattered documentation—this is tedious and prone to inconsistency, especially when notes are incomplete or span multiple care settings.

A scalable triage‑and‑summarize workflow could free infection prevention and stewardship staff to focus on higher‑value tasks (antimicrobial optimization, interventions) within constrained staffing and compute budgets.

What They Did

Retrospective VA cohort of CIED procedures from July 1, 2021 to September 30, 2023: 12,927 procedures across 71 VA centers; applied a previously validated ML trigger to structured VA and Community Care data and flagged 334 cases with ≥10% predicted infection probability.

From the flagged cases, selected a weighted, stratified random sample of 100 charts (50 confirmed infections, 50 confirmed non‑infections as determined by manual review) for LLM‑assisted evaluation.

Manually copied relevant clinical notes into a HIPAA‑compliant VA LLM (VA GPT using GPT‑4o, temperature=0), iteratively refined prompts to extract infection elements and timelines, and asked the LLM to apply NHSN‑aligned definitions.

Compared GenAI outputs to expert manual chart review (manual review used as the gold standard), recorded per‑item concordance and error types, and measured time for each review step (including the note harvesting and data entry steps used in this evaluation).

What They Found

In the 100‑case, stratified sample the LLM correctly identified all 50 sampled infection cases and all 50 sampled non‑infection cases (100% sensitivity and 100% specificity for infection detection within this enriched test set). Note: this performance reflects a deliberately enriched test set and retrospective design, not real‑world prevalence.

For procedure‑attribution (deciding if an infection occurred within the 90‑day window and was procedure‑related), the authors report overall sensitivity ≈82% (95% CI: 68.6%–91.4%) and specificity 100% in the stratified sample. Reported counts for timing/attribution vary across analyses in the manuscript (for example, the paper reports GenAI attributed 39 of 46 procedure‑related infections in one analysis and created accurate timelines in 42 of 50 infection cases in another), but the central finding is consistent: timeline/date sequencing was the main source of discordance and improved with iterative prompt revisions (procedure‑date extraction was reported to improve from 43/50 to 50/50 after prompt updates).

Element‑level extraction was strong: symptoms, clinician diagnosis, procedural interventions, and antibiotic therapy were concordant with manual review in essentially all sampled infection cases; infection type matched in 49/50 (98%) and culture dates in 49/50 (98%), with the single culture‑date miss due to an outside‑system culture not present in VA notes.

Workflow impact: the GenAI‑assisted process averaged ~5–7 minutes per chart versus ~25 minutes for full manual review in this study (including manual note extraction/copy‑paste). The time savings derived from the LLM doing the initial reading and summarization, but human verification—especially for dates and attribution—remained necessary.

Takeaways

Treat LLMs as advanced summarizers and extractors in a human‑in‑the‑loop surveillance workflow, not as autonomous adjudicators—humans must verify dates and attribution before reporting.

Design surveillance as triage‑first: use an ML pre‑filter to narrow the corpus, then apply an LLM to summarize and extract; this concentrates compute and reviewer effort where it’s most needed but requires local validation and prompt calibration.

Assess data completeness and fit locally: timeline accuracy depends on access to outside‑facility records and microbiology data, so validate performance on local edge cases and ensure the AI can see the same data that manual reviewers use.

For pharmacists: adopt the LLM as a 'smart intake' to prioritize reviews and accelerate stewardship actions, but define adjudication roles, maintain governance and audit trails, and periodically revalidate prompts and performance.

Strengths and Limitations

Strengths:

Operational, multi‑center VA evaluation combining an ML pre‑filter with a HIPAA‑compliant LLM (VA GPT/GPT‑4o) and iterative prompt refinement applied to real clinical notes across 71 centers.

Rigorous element‑level abstraction against manual, NHSN‑aligned review with timed workflow metrics and qualitative error categorization.

Limitations:

Single‑system VA cohort (predominantly older male veterans) and VA documentation patterns limit generalizability to other hospitals, EHRs, and patient populations.; Retrospective design with ML pre‑filtering and a stratified 50/50 sampled test set enriched for infections likely inflates apparent accuracy; the LLM was not evaluated on an unfiltered, prospective dataset or externally validated.

Some relevant data (e.g., outside‑system clinical notes or external culture dates) were unavailable to the LLM in this study, producing unavoidable misses; the study also relied on manual note harvesting and copy‑paste, which reduces immediate automation feasibility.

The manuscript reports slightly different denominators across analyses for timing/attribution (e.g., 39/46, 42/50), so performance for timeline/attribution should be interpreted cautiously and confirmed prospectively.

Bottom Line

This study demonstrates feasibility of a hybrid ML + LLM surveillance pipeline under retrospective, pre‑filtered conditions: LLMs can substantially reduce chart‑reading time and reliably detect documented CIED infections in an enriched sample, but human adjudication, local validation (especially for timelines and external data coverage), prospective evaluation, and governance are required before operational deployment.