--- title: "Scale AI just published the benchmark that prices its own Qatar contracts" description: "Scale AI's new Singapore legal benchmark measures exactly the gap that its Qatar contracts, signed three days earlier, pay Scale to close. Oracle's sovereign-AI story and private legal-AI valuations now have a number to argue with." publisher: "The Inference" section: "Model" published: 2026-11-13T15:21:47.855Z modified: 2026-11-13T15:21:47.855Z canonical: https://theinference.org/article/scale-ai-just-published-the-benchmark-that-prices-its-own-qatar-contracts language: en keywords: "Scale AI, AI Benchmarks, Legal AI, Qatar, Oracle" --- # Scale AI just published the benchmark that prices its own Qatar contracts > Scale AI's new Singapore legal benchmark measures exactly the gap that its Qatar contracts, signed three days earlier, pay Scale to close. Oracle's sovereign-AI story and private legal-AI valuations now have a number to argue with. ## A test that prices the gap On November 12, [Scale AI](https://theinference.org/markets/companies/scale-ai) and Singapore's Infocomm Media Development Authority (IMDA) published [SG-LegalClarify, a benchmark](https://www.unite.ai/scale-ai-launches-singapore-legal-reasoning-benchmark-with-imda/) putting 19 language models through 1,860 scenarios across ten areas of Singapore law, graded against 12,546 lawyer-written criteria [1]. Twenty-three verified lawyers wrote the 1,100 source questions [1]. The result is the first dated price tag on AI doing legal work, and it landed three days after Scale sold Qatar two more AI-for-justice contracts. ## The score that should worry buyers Even when models gathered every fact lawyers flagged as relevant, 55.6% reached the correct legal conclusion, 32.1% reached an incorrect one and 12.3% gave none [1]. Ask a model a question missing a material fact and it stays silent about 39% of the time; when it does ask, it requests only 39% of the facts experts identified [1]. Rule recall was the weakest skill, at 43.2% closed-book [1]. Claude Fable 5 led closed-book at 69.8% of mandatory criteria; with web search, Kimi K3 led at 77.8% [1]. *Chart: **When models ask for every missing fact, 32.1% still reach the wrong legal answer** Bar chart of legal outcomes when models asked for every missing fact: 55.6% correct, 32.1% incorrect, 12.3% no conclusion* *Outcome of legal questions among trials where models requested every relevant missing fact, % of trials. Source: Scale AI/IMDA SG-LegalClarify benchmark, November 12, 2026. [1]* ## What Scale sold while the ink dried On November 10 at MWC Doha, Qatar's communications and justice ministries signed [two agreements with Scale AI](https://thepeninsulaqatar.com/article/10/11/2026/mcit-moj-sign-two-agreements-with-scale-ai-to-develop-solutions-for-justice-sector): SAK for notarisation and document authentication, Shoor for state litigation and legal research, under a five-year partnership targeting more than 50 government AI projects by 2029 [2] [3]. Qatar's earlier Smart Legislative Advisor, built with the same three parties, aims to cut legislative review from about 35 days to five [3]. On November 9, Scale and [Oracle](https://theinference.org/markets/companies/oracle) published [a blueprint for keeping government AI workloads in-country](https://superpowerdaily.com/posts/scale-ai-and-oracle-publish-a-blueprint-for-keeping-government-ai-workloads-in-country), drawing on Scale's Qatar deployments [4]. ## The referee sells the game One company now sells governments the AI workflow and [the measurement of that workflow](https://theinference.org/article/scale-ai-cut-89-tasks-from-its-coding-benchmark-after-an-audit-found-gaming-then-graded-its-own-fix), and the benchmark prices the human-in-the-loop gap that closing is what Scale gets paid for [1] [2]. Scale is private; the listed read-through is Oracle (ORCL), whose sovereign-AI pitch leans on Scale's government record, plus [private legal-AI marks](https://theinference.org/article/harvey-built-the-legal-ai-exam-z-ai-just-passed-19-of-it-at-about-15-cents-a-task) that now have a benchmark to price seats against. Watch for a government buyer, not a vendor, publishing its own independent evaluation; that would break the loop. Deep dive in the sources below. ## Takeaway Scale AI now sells governments the legal AI workflow and the test that grades it, and its own benchmark says models still reach the wrong legal answer a third of the time. ## Sources 1. [Unite.AI, "Scale AI Launches Singapore Legal Reasoning Benchmark with IMDA", 12 November 2026](https://www.unite.ai/scale-ai-launches-singapore-legal-reasoning-benchmark-with-imda/) 2. [Tech Revolt, "Scale AI Enters Qatar's Justice System as AI Moves Into High-Stakes Government Work", 13 November 2026](https://www.techrevolt.org/articles/scale-ai-enters-qatar-s-justice-system-as-ai-moves-into-high-stakes-government-work) 3. [The Peninsula Qatar, "MCIT, MOJ sign two agreements with Scale AI to develop solutions for justice sector", 10 November 2026](https://thepeninsulaqatar.com/article/10/11/2026/mcit-moj-sign-two-agreements-with-scale-ai-to-develop-solutions-for-justice-sector) 4. [Superpower Daily, "Scale AI and Oracle publish a blueprint for keeping government AI workloads in-country", 9 November 2026](https://superpowerdaily.com/posts/scale-ai-and-oracle-publish-a-blueprint-for-keeping-government-ai-workloads-in-country) --- Scale AI just published the benchmark that prices its own Qatar contracts — The Inference. Canonical: https://theinference.org/article/scale-ai-just-published-the-benchmark-that-prices-its-own-qatar-contracts