Clinical Notes Power Early Neonatal Risk Triage

Predictive Model DevelopmentPredictive External Validation

Authors:Feng Xie; Philip Chung; Jonathan D Reiss, et al.

Journal:The Lancet Digital Health

DOI:10.1016/j.landig.2025.100926

Reactive alerts often miss early warning signs hidden in notes. This AI helps pharmacists prioritize high-risk infants for stewardship.

December 18, 2025

Quick Take

NeonatalBERT, a foundation model pre-trained specifically on neonatal clinical notes, significantly outperformed standard tabular risk scores for predicting severe neonatal morbidities. In a primary cohort of over 32,000 infants, the model achieved a mean AUPRC of 0.291 versus 0.194 for tabular baselines, effectively extracting signal from the unstructured "dark data" of the EHR.

While the high accuracy (AUROC >0.90) is promising, the modest precision (AUPRC ~0.29) indicates that for every ten alerts, roughly seven are false positives. This performance profile positions the tool as a powerful "second pair of eyes" for worklist prioritization and stewardship huddles, but it lacks the precision required for automated clinical decision-making or auto-verification.

Why it Matters

Approximately 80% of clinical data resides in unstructured notes, yet pharmacy workflows remain heavily dependent on structured triggers like lab values or order sets. This study proves that "reading" the notes can identify deterioration signals (e.g., subtle nursing observations of lethargy or feeding intolerance) hours or days before structured data thresholds are breached.

For high-stakes conditions like culture-negative sepsis and bronchopulmonary dysplasia (BPD), diagnostic ambiguity drives significant resource utilization and antibiotic exposure. A note-based risk score offers an objective "tie-breaker" to empower antimicrobial stewardship programs to stop antibiotics earlier or to identify high-risk infants for targeted BPD prophylaxis (e.g., DART protocol).

Operationalizing this level of intelligence requires a shift from reactive to proactive pharmacy practice, but it comes with a steep infrastructure cost. Real-time inference on clinical notes requires GPU architecture and advanced EHR integration (FHIR), presenting a new "build vs. buy" challenge for pharmacy informatics leaders.

What They Did

The researchers developed NeonatalBERT by conducting domain-adaptive pre-training of Bio-ClinicalBERT on millions of sentences from neonatal notes, effectively teaching the model the unique "dialect" of the NICU.

They fine-tuned the model to predict 19 outcomes using notes from the first 24 hours of life in a primary cohort from Stanford (n=32,321) and validated it on an external, more acute cohort from Beth Israel Deaconess Medical Center (n=7,061).

The study benchmarked NeonatalBERT against rigorous baselines, including logistic regression on tabular data (vitals/labs) and general-purpose language models (including Llama 3.1). They specifically analyzed performance on distinct note types (nursing, physician, consults) to determine where the predictive signal resides.

What They Found

Unstructured text provided a massive lift over structured data for complex, evolving conditions. The model outperformed tabular baselines by approximately 50% in terms of precision-recall (AUPRC), showing the highest value for conditions like pulmonary hemorrhage (88x lift over random chance), BPD, and sepsis.

The model offered little to no value for lab-driven conditions. For jaundice, the text-based model performed comparably to simple tabular regression, confirming that AI should be targeted at "gray zone" diagnoses rather than conditions with clear biomarker definitions.

Generative AI (Llama 3.1) performed poorly compared to the specialized BERT model for risk prediction. This suggests that for reliable risk scoring, discriminative "reader" models are currently superior to the "generative" chatbots that dominate the current AI hype cycle.

Performance improved as documentation accumulated. Predictive power increased from Day 1 to Day 7, suggesting the model acts as a dynamic learning system that refines its risk assessment as the clinical narrative evolves.

Takeaways

Use this as a prioritization engine, not a diagnostic tool. With a positive predictive value around 30%, this tool is best suited to populate a "High Risk Watch List" for pharmacists to review during rounds, ensuring the sickest babies get human attention first.

Focus pilot efforts on "Gray Zone" diseases. Do not waste resources deploying this for hyperbilirubinemia or electrolyte management. Target areas where clinical uncertainty drives variation, such as duration of therapy for culture-negative sepsis or patient selection for steroid prophylaxis in BPD.

Beware the infrastructure requirements. Unlike a standard clinical decision support rule, this model requires feeding live text to a GPU inference server. Before committing to adoption, audit your institution's ability to access real-time note text via FHIR and your capacity to govern a model that is subject to "drift" if documentation styles change.

Establish governance before deployment. A "High Risk" alert creates an information hazard if there is no protocol attached. Pharmacy and Therapeutics committees must define the intervention (e.g., "Pharmacist to review for stewardship eligibility") before the alert is turned on to prevent alert fatigue and anxiety-driven over-treatment.

Strengths and Limitations

Strengths:

The study is notable for its scale and external validation, testing the model on a distinct population (BIDMC) with higher acuity than the training set, which suggests genuine generalizability rather than site-specific overfitting.

The use of diverse note types, specifically nursing notes, acknowledges that frontline observations often capture subtle physiological changes before physician progress notes or structured data, validating the "interdisciplinary" nature of the data.

Limitations:

The model weights have not been publicly released due to privacy concerns, meaning health systems cannot simply "plug and play" but would likely need to replicate the costly training process locally.

The study did not compare NeonatalBERT against existing gold-standard clinical calculators (like the Kaiser Sepsis Calculator), leaving it unclear if the heavy compute cost yields better outcomes than free, widely used tools for early-onset sepsis.

Disease labels were derived from administrative codes (ICD/SNOMED) rather than adjudicated clinical criteria. This creates a risk of "label leakage," where the model might be predicting a physician's suspicion (documented in the note) rather than the actual physiological onset of the disease.

Bottom Line

NeonatalBERT provides compelling evidence that the next frontier of pharmacy informatics lies in unstructured text analysis, offering a significant upgrade over traditional rule-based alerts for complex neonatal conditions; however, the current precision levels and infrastructure demands suggest this is a technology to monitor and pilot in sophisticated academic centers rather than a ready-to-deploy commercial solution.