Oncology AI Co‑Pilot Drafts Breast Cancer Regimens, With Caveats

• AMIE, an oncology‑focused large language model (LLM) assistant, generated guideline‑concordant breast‑cancer management plans in ~90% of 60 synthetic cases and was judged not harmful in 92% of responses.

• Adjuvant chemotherapy decisions were accurate in ~98% of cases; summaries captured required clinical information in ~95% and molecular alignment was strong (~88%).

• More than 75% of AMIE’s plans were considered implementable without modification, suggesting utility for drafting oncology summaries and order templates for pharmacist review.

• Key failure modes remain: AMIE under‑sequenced or suggested outdated adjuvant approaches in ~20–25% of refractory cases, so attending‑level verification is still required.

• Breast oncology is common and therapeutically complex; clear, guideline‑aligned plans are essential for safe chemotherapy, endocrine, and targeted therapy verification by pharmacy teams.

• Subspecialist shortages and growing regimen complexity create operational strain on inpatient and outpatient pharmacy verification workflows where reliable draft plans could improve throughput.

• Understanding where LLM assistants are reliable versus error‑prone is necessary to design clinical decision support (CDS) that augments pharmacy capacity while protecting patient safety.

• Curated 60 synthetic breast cancer vignettes (30 treatment‑naive, 20 refractory, 10 rare histologies) checked for clinical plausibility by breast oncologists and standardized for use across respondents.

• Deployed AMIE (Gemini 2.5 Pro) with inference‑time web retrieval and a multistage self‑critique pipeline to produce case summaries and management plans; AMIE was not fine‑tuned to breast oncology and did not output dosing schedules.

• Compared AMIE to two internal medicine trainees, two oncology fellows, and two general oncology attendings; five breast oncologist evaluators rated all responses using a 19‑item rubric aligned to NCCN guidelines.

• Conducted a retrospective, simulated evaluation focused on guideline concordance, safety, summarization, and molecular alignment; the study did not test integration with local EHRs, formularies, dosing calculators, or real‑time inpatient data.

• AMIE produced guideline‑concordant management recommendations in >90% of the 60 synthetic cases, with particularly strong performance for neoadjuvant planning and adjuvant chemotherapy decisions (~0.91 and 0.98 favorable rates respectively).

• Safety and output quality were high: 92% of responses rated not harmful, ~98% were free of clinically significant hallucinations, summaries included necessary information in ~95%, and molecular / genetic alignment scored ~0.88.

• Over 75% of AMIE’s plans were judged implementable without modification, indicating operational potential to pre‑draft chemotherapy/hormonal order sets and structured summaries for pharmacist review.

• Performance improvements correlated with the inference‑time web retrieval plus multistage self‑critique pipeline; however, AMIE showed undertreatment or outdated adjuvant sequencing in ~20–25% of refractory cases—an explicit verification risk for pharmacy teams.

• LLM assistants like AMIE are most promising as front‑end tools that assemble structured breast cancer summaries and outline guideline‑concordant plans for many treatment‑naive cases, reducing administrative burden.

• For pharmacists, AMIE behaves like a capable junior colleague: often matching or exceeding trainee reasoning on surgery, radiation, and many adjuvant choices, but still requiring specialist review before clinical action.

• Refractory and rare‑histology cases are the primary danger zone—AI suggestions in these scenarios demand heightened skepticism and mandatory attending or specialist pharmacist verification.

• Because the study used sanitized synthetic vignettes and omitted dosing schedules, near‑term impact is likely in summarization, education, and guideline‑aware second opinions rather than autonomous chemotherapy order entry.

Strengths:

• Three breast oncologists curated 60 clinically representative synthetic cases across treatment‑naive, refractory, and rare histologies with independent plausibility review.

• Five breast‑oncology specialists applied a pilot‑tested 19‑item rubric aligned to NCCN guidance to evaluate safety, sequencing, molecular alignment, hallucinations, supportive care, and summarization.

Limitations:

• Evaluation used simplified, sanitized text vignettes and high‑level regimens without dosing, toxicity management, or longitudinal follow‑up, limiting applicability to real‑world inpatient pharmacy workflows.

• Single‑center case design, modest scenario set, reliance on web retrieval for up‑to‑date guidance, and limited interrater agreement on adjuvant therapy and harm constrain generalizability.

AMIE demonstrates strong potential as a breast‑oncology co‑pilot for drafting guideline‑aware summaries and management outlines, but it remains a research tool that requires attending‑level oversight and prospective, real‑world validation—particularly before any use in chemotherapy order verification.