Co-Annotator: Expert-Distilled ViT and VLM for Visual and Documentation Guidance in Age-Related Macular Degeneration

Clinical AI often optimizes predictive performance without engaging how clinicians decide where to look and what to write. We present Co-Annotator, which distills expert gaze and dictation into two guidance components: a gaze-aligned Vision Transformer producing fixation-aligned areas of interest (AOIs), and an ontology-bounded vision-language model (VLM) that pre-fills editable biomarker summaries for retinal optical coherence tomography (OCT). We first collect expert gaze and dictations (US1) to train the models, significantly improving diagnostic accuracy and biomarker generation. We then deploy the system with ophthalmology residents: a controlled resident study (US2) confirmed each modality is safe and independently beneficial, with AOI guidance producing lasting perceptual efficiency gains through post-guidance carryover and VLM guidance more than doubling biomarker documentation breadth. In a combined deployment across two academic institutions (US3), providing both modalities simultaneously produced efficiency gains that substantially exceeded either modality alone: correct diagnoses per minute increased by 40% and comment editing time fell by 67%, without compromising diagnostic accuracy. Notably, neither modality improved efficiency during guidance in US2, which makes the in-guidance efficiency gain under combined guidance in US3 the more striking result. Expert-distilled multimodal guidance can remove two distinct clinical workflow bottlenecks at once (visual search overhead and documentation burden) without compromising the diagnostic accuracy clinicians already achieve.

Large language models improve physician accuracy but lead to false reliance

Retrieval-augmented large language models (LLMs) promise source-linked clinical support, but their value depends on whether displayed evidence guides rather than distorts physician reliance. We developed CORA, an agentic retrieval-augmented LLM, to investigate how source-linked assistance affects physician decision-making. CORA maintained benchmark performance and achieved larger gains on cases published after the models' training-data cutoffs. In a study of 46 physicians, accuracy increased from 70.8% unaided to 82.6% with CORA. Supporting citations predicted correct answers (87.7% vs 65.5%), but citations created an important asymmetry: perceived support increased adoption of correct advice from 34% to 76.9% but when an incorrect LLM answer appeared citation-supported, physician resistance to it fell from 92% to 34.8%. These findings show that source-linked LLM assistance can improve physician accuracy while introducing a grounding-dependent safety risk.

The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing

Autonomous AI systems are transitioning from advisory roles to autonomous ones for medication prescriptions. Recent U.S. bill H.R. 238 and Utah's prescription-renewal pilot program both authorize AI to prescribe medications in an agentic capacity. While many regulatory guidelines suggest aggregate model performance metrics at the point of clearance, they do not require i) calibrated per-prediction confidence for action-gated thresholds, ii) differentiated communication between uncertainty arising from model ignorance (epistemic) from genuine clinical ambiguity (aleatoric), and iii) inferential transparency at the moment of decision enabling liability allocation. Here, we argue these three architectural features are minimum conditions for safe autonomous prescribing, and validate them with a survey of 136 U.S. prescribing clinicians. Our results suggest prescribing clinicians i) would not permit autonomous prescribing without a confidence-based escalation mechanism, ii) preferred a competing-options summary for aleatoric uncertainty but preferred abstention for epistemic uncertainty, and iii) were only willing to accept liability when inferential transparency enabled them to make a decision under acknowledged uncertainty. These findings indicate that our recommended architectural features would encourage higher rates of clinician adoption of autonomous AI prescribing, largely through collapsing much of what "autonomy" conventionally means.