Quick Take
- NEJM AI piloted an invitation-only Fast Track using GPT-5 (with Thinking) and Google Gemini 2.5 Pro to generate advisory reviews inside the journal’s closed editorial workflow. - The pilot aimed to return decisions within 7 days for high-likelihood accept submissions; AI acted solely as an informed critique while humans retained final decision-making authority. - For inpatient pharmacy, this offers a practical template to speed and standardize Pharmacy & Therapeutics (P&T) screening, residency project QA, and operational protocol review—provided strict oversight is in place.
Why it Matters
- Operational Efficiency: Standardized prompts and vetted workflows can speed up protocol reviews, Medication Use Evaluations (MUEs), and formulary drafting. However, this creates a bottleneck if the AI generates excessive ideas or requires heavy fact-checking, causing pharmacists to spend more time validating the output than they saved on the initial draft.
- Evidence Quality & Equity: Structured first-pass AI reviews may surface methodological flaws earlier. However, AI critiques can reinforce narrow metrics or bias decisions toward flagship vendors. Hallucinated or mis-summarized evidence could corrupt decision if not caught.
- IT & Contract Risk: While clarifying vendor data rights is essential, the "access gap" poses the greater strategic threat. Failing to provide a sanctioned enterprise tool creates a vacuum where staff inevitably default to personal accounts, leaking sensitive data into unmonitored public models.
Bottom Line
Human+AI review can accelerate and harmonize pharmacy evidence workflows without ceding accountability—but only under disciplined pilot that enforce human sign-off, tight structure, and auditable logs.
Key Details
- The Workflow: A human handling editor first wrote an independent review, then submitted the manuscript and a structured prompt to two LLMs. The output included a summary, ordered major/minor comments, and a brief recommendation.
- Statistical Protocol: The team used a six-step human–AI protocol: read documents, upload materials/guidelines, ask sequential questions, draft a review, solicit AI critique, and finalize.
- What AI Caught: In one example, the AI flagged that a "work‑outside‑work" result lost significance after removing the top 3% of observations, prompting necessary manuscript revisions.
- What AI Missed: The models occasionally gave off-topic suggestions or missed nuances in complex designs (e.g., mediation with stepped‑wedge trials), reinforcing that AI cannot replace human statistical expertise.
- Transparency: With authors’ consent, NEJM AI published the AI prompts, human and AI reviews, and full statistical chat transcripts, creating an auditable trail.