Health

OpenEvidence's prior-auth letters no better than ChatGPT's in Mayo test; $15 billion buys its distribution

A Mayo Clinic head-to-head and a Nature Medicine study put the quality of OpenEvidence's answers on the record in one week: good enough to use, no longer uniquely good. At $15 billion, investors are paying for distribution, and distribution now has company.

Vincent JiangVincent Jiangthree-minute read
Share
The OpenEvidence answer engine's home screen, showing the search bar clinicians use to ask medical questions
1 / 6Slide 1 of 6
OpenEvidence's home screen. The tool that 65% of US physicians use was tested head to head against ChatGPT-5 in a Mayo Clinic-led prior authorization study.

The most valuable document in American medicine is short, unglamorous, and decides whether an insurer pays for a drug: the prior authorization letter. A Mayo Clinic-led team tested who drafts it better, OpenEvidence~$6B — OpenEvidence, private, latest valuation ~$6B or ChatGPT-5, across 28 nephrology scenarios at ASN Kidney Week in Denver this week 1.

OpenEvidence, the AI answer engine 65% of US physicians use, put false statements in 7.1% of its letters against 3.6% for ChatGPT-5, and its clinical reasoning was rated strong in 78.6% of letters against 89.3% 1. The ICD-10 coding was correct in 67.9% of OpenEvidence letters against 78.6% for ChatGPT-5, and the Mayo authors tied that gap to length: OpenEvidence wrote longer letters that often blew past the 350-word target, and more words mean more chances to be wrong 1.

OpenEvidence trailed ChatGPT-5 on every letter-quality measure, none by a significant margin

  • OpenEvidence
  • ChatGPT-5
050100False statementsStrong clinical reasoningCorrect ICD-10 coding67.978.6
Data
OpenEvidenceChatGPT-5
False statements7.13.6
Strong clinical reasoning78.689.3
Correct ICD-10 coding67.978.6
Share of 28 nephrology prior authorization letters meeting each measure, Mayo Clinic-led test presented at ASN Kidney Week 2026. No gap reached statistical significance.1

No gap reached statistical significance, and the authors' conclusion was measured: these are drafts for a clinician to review, not finished letters. Both tools hallucinated 1.

The stakes: a denial that is winnable and abandoned

Medicare Advantage insurers made 52.8 million prior authorization determinations in 2024, denied 4.1 million (7.7%), and only 11.5% of denials were appealed 210. When clinicians did appeal, 80.7% were fully or partially overturned 210. The denial is routinely winnable and routinely abandoned, which is precisely the workflow AI letter-writing is being sold into 2.

The valuation buys distribution, and the accuracy record now says so

OpenEvidence reached $15 billion in September after a $250 million round co-led by Andreessen Horowitz and Byers Capital, its third markup this year, on a projected $400-500 million in 2026 advertising revenue from pharmaceutical companies 3. What the money buys is distribution: 952,000 clinicians had submitted 271 million queries by June 2026, monthly actives hit 703,265, and the platform handled 42 million prompts in October 311.

What it does not buy, on the published record, is the best model. In a June Nature Medicine study OpenEvidence scored 89.6% on 500 MedQA questions, below Gemini 3.1 Pro at 97.4% and GPT-5.2 at 94.2%; blinded clinicians rated it level with UpToDate Expert AI and Google AI Overview 4.

On MedQA, OpenEvidence trailed Gemini 3.1 Pro by 7.8 points

0%20%40%60%80%100%OpenEvidence89.6%Claude Opus 4.690.2%GPT-5.294.2%Gemini 3.1 Pro97.4%
Data
Value
OpenEvidence89.6%
Claude Opus 4.690.2%
GPT-5.294.2%
Gemini 3.1 Pro97.4%
Accuracy on the same 500 MedQA questions, Nature Medicine study, June 2026.4

The moat is the network, and this week its edges got crowded

Memorial Sloan Kettering is wiring its OncoKB tumor-genetics database into OpenEvidence and Epic 7. OneOncology, whose community oncologists treat 1.5 million patients, embedded the tool in its workflows 5.

Meanwhile Doximity−2.02% — Doximity, down 2.02 percent today, which reaches more than 85% of US doctors, settled the lawsuit OpenEvidence filed against it in June 2025, all claims dismissed with prejudice on October 17, each side bearing its own costs 8912. In that suit OpenEvidence alleged that Doximity staff posed as doctors and used planted prompt-injection instructions to trick the tool into revealing its confidential system prompts 89.

Two days after the settlement, Doximity put Wolters Kluwer's UpToDate, 13,000 expert-reviewed topics, inside its rival Ask tool, and the stock rose 7.3% 8912. The company that owns the licensed content now sits on both sides of the trade 8912.

The usage study itself concedes the open question: it counts who asks, not whether the answers are right or whether patients do better. Treatment and management questions, 33.1% of physician queries, outnumber diagnostic ones at 13.9%, so the unmeasured surface is the widest one 613.

The owners of licensed content are the investable exposure

The scarce assets in evidence-based medicine are licensed content and the physician workflow, and the owners of both just moved against the private mark. The conflict to watch sits between the advertising line and the answers: if pharma money inside the tool starts shaping what doctors read, the first signs would come from disclosures around the $400-500 million 2026 ad-revenue projection, whose next test is the year-end reporting season, or from a clinician or publisher challenging the ad-and-answer model in court or in a study 3.

Until then the money is on distribution, and the quality of the answers is now on the record.

Get Health by email, free

AI in medicine, pharma and biotech. In our daily email, every weekday.

Loading DOCS

Reader comments

0 comments

    More in Health

    All in this topic →

    Health

    Tempus AI beat Q3, but $80 million of 2027 growth rides on a December 31 Medicare deadline

    Q3 revenue of $411.5 million beat consensus, but the quarterly letter books about $80 million of 2027 growth on a Medicare repricing that only pays if the xT conversion finishes by December 31, while the cheapest product, MRD testing, grows fastest and drags reimbursement down.

    Health

    Eli Lilly puts Foundayo at $149 a month and Zepbound at $299 on Hims & Hers and LifeMD, self-pay

    Two distribution deals in two days put Lilly's Zepbound and Foundayo on Hims & Hers and LifeMD at self-pay prices starting at $149 a month. Lilly keeps the price sheet and the margin; the platforms supply the funnel.

    Health

    Novo Nordisk's 27% profit jump is the adjusted number; the reported number fell 9%

    Novo's beat-and-raise quarter hides a flat underlying US market propped up by a temporary Medicare bridge, and a reported profit that fell 9% while the adjusted line rose 27%. The CFO who called the write-offs one-offs leaves on December 31.

    Health

    Five hospitals say Lilly cut off tens of millions in 340B discounts until they hand over patient data

    Five Massachusetts hospitals say Eli Lilly cut off tens of millions in 340B drug discounts until they surrender patient claims data to a private-equity-owned platform. Lilly calls it fraud policing, and the same gross-to-net machinery under dispute just produced a record $24.4 billion quarter.

    Health

    Retatrutide cut knee pain up to 75.8% in Phase 3, and the fine print matters

    Lilly's first pivotal data for its triple-agonist drug lands at ACR this weekend just as Novo halves Wegovy's list price. The race has split between outcomes and price, and Lilly's headline percentages were never pre-specified.

    Health

    Ambience's KLAS validation put $37,500 a year on each ER doctor; $6.69 billion Abridge sells minutes

    A KLAS validation published November 3 puts about $37,500 of extra annual productivity on each emergency physician at Memorial Hermann, the Houston-based health system, the largest per-provider dollar figure KLAS has validated to date. Abridge, marked at $6.69 billion, has published minutes saved per clinician-day and no validated dollars.

    Get the newsletter

    The day's AI and market news, in your inbox every weekday.