Meta's Muse models top the new Scale AI benchmark; the referee is 49% Meta-owned
Scale AI, 49%-owned by Meta since a $14.3 billion deal in 2025, has published a benchmark that ranks Meta's Muse models first, second and third. OpenAI, Google and Anthropic all declined to endorse it.
Vincent Jiang · 3 min read
What Meta's $14.3 billion actually bought
On June 30, 2025, Meta+0.18% — Meta, up 0.18 percent today paid $14.3 billion for 49% of Scale AI$29B — Scale AI, private, latest valuation $29B, valuing the data-and-evaluation company near $29 billion, and hired co-founder Alexandr Wang the same day as chief AI officer to run Meta Superintelligence Labs 1. Wang stepped down as Scale's chief executive but stayed on as chairman 2. What Meta left behind is the industry's grader: Scale's own release says its data engine underpins model development at most major frontier labs, serving the U.S. government and the Fortune 500 2.
The grader's leaderboard puts the owner's models first
On October 8, Scale published DistressBench: 718 crisis conversations written by 19 licensed clinicians and counselors, scored against 6,901 clinician-weighted criteria 34. Pooled across 25 frontier models, in 35.3% of conversations where a model met every criterion for recognizing a suicide or self-harm crisis, it still failed to route the user to human help 3. The best model delivered 88.4% of required care, the median 71.4%, against 98.7% for clinician reference replies 3. The top three slots on the leaderboard belong to Meta's Muse models 3.
No model comes close to the care clinicians deliver
Data
| Share of required care delivered | |
|---|---|
| Best model (Muse Spark 1.2) | 88.4% |
| Median frontier model | 71.4% |
| Clinician reference replies | 98.7% |
Meta's Muse models hold the top three slots on Scale AI's DistressBench
Data
| Value | |
|---|---|
| Muse Spark 1.2 (Meta) | 88.39% |
| Muse Spark 1.3 (Meta) | 84.88% |
| Muse Glimmer 30B (Meta) | 80.57% |
| GPT-5.6 Terra (OpenAI) | 76.44% |
| Claude Opus 5.5 (Anthropic) | 75.58% |
| Gemini 3.8 Flash (Google) | 70.74% |
OpenAI, Google and Anthropic declined to endorse it
OpenAI$1.18T — OpenAI, private, latest valuation $1.18T said it "did not have sufficient evidence to fully evaluate" the study; Google declined to comment; Anthropic$350B — Anthropic, private, latest valuation $350B did not respond 4. The method is disclosed in unusual detail: a three-judge panel drawn from Anthropic, Google+0.49% — Google, up 0.49 percent today and Meta models, selected from 455 possible combinations against 2,908 clinician labels, agreeing with clinicians at Cohen's κ of 0.732 3. Scale itself warns a high score "does not license deployment as a mental-health product," and the benchmark is English-only, built around U.S. crisis lines 3.
The CEO story in circulation is a year old
A report making the rounds this week says Scale will elevate strategy chief Jason Droege to chief executive as Wang departs for Meta 5. That describes June 2025: Droege ran Scale as interim CEO from then, and Scale's July 30, 2026, announcement names Francis deSouza, late of Google Cloud and Illumina+0.74% — Illumina, up 0.74 percent today, chief executive effective August 10 2. The genuinely new deal this week was sovereign: an MoU with Presight, Abu Dhabi's sovereign-AI operator, signed October 9 to evaluate and govern agents for Gulf power, water, healthcare and transport systems 6.
Watch whether rivals keep paying the referee
Zuckerberg said on October 12 that Meta weighed a $100–200 billion commitment in late 2025 before its models had proven themselves, with CFO Susan Li warning Wang that failure would be "really, really bad" in financial terms; 2026 capex guidance stands at $130–145 billion 7. Muse, launched September 8, drew more than 2.5 million downloads in about two weeks and an 11% single-session jump, adding roughly $200 billion of market value 8. The market has priced Meta's side of the bet. Whether OpenAI, Google and Anthropic keep submitting to a referee their competitor part-owns is the tell.
Deepdive
AI-generated from this story and its cited sources. Not investment advice.
