--- title: "China's free AI models trail Claude by 26 points on secret tests; Alibaba's Qwen may not stay free" description: "Artificial Analysis moved 60 percent of its benchmark onto secret tests, and the best downloadable Chinese models now trail Anthropic's Claude by 26 points. Alibaba's Qwen 4, promised for this week, still has no license on the table, just as Anthropic readies a Nasdaq listing at $17 billion a quarter." publisher: "The Inference" section: "Model" published: 2026-11-03T15:13:23.400Z modified: 2026-11-03T15:13:23.400Z canonical: https://theinference.org/article/china-s-free-ai-models-trail-claude-by-26-points-on-secret-tests-alibaba-s-qwen-may-not-stay-free language: en keywords: "AI Models, Open Weights, Benchmarks, Alibaba, Anthropic" --- # China's free AI models trail Claude by 26 points on secret tests; Alibaba's Qwen may not stay free > Artificial Analysis moved 60 percent of its benchmark onto secret tests, and the best downloadable Chinese models now trail Anthropic's Claude by 26 points. Alibaba's Qwen 4, promised for this week, still has no license on the table, just as Anthropic readies a Nasdaq listing at $17 billion a quarter. ## The leaderboard went private, and the gap quadrupled Artificial Analysis, the independent firm whose index is AI's most-cited scorecard, [rebuilt its exam on October 27](https://artificialanalysis.ai/) [1][4]. Sixty percent of the score now rests on secret, held-out questions, up from 45 percent, so labs can no longer train to the test [1][4]. The gap between the best open-weight model and the leader widened overnight from 3 points to 12 [1]. Moonshot AI's Kimi K3.1 Pro, fourth in the world on the old index, fell to ninth [2]. It scores 46, but its weights are not downloadable until November 4 [3][5]. Among weights a developer can actually download, the fall is steeper. DeepSeek's V4.1 Flash and Zhipu's GLM-5.3 lead at 32 points, 26 behind Anthropic's Claude Opus 5.5 at 58 [3][4]. Claude Haiku 5.5, [Anthropic](https://theinference.org/markets/companies/anthropic)'s smallest model, scores 36 and beats every downloadable release [5]. A full day of [DeepSeek](https://theinference.org/markets/companies/deepseek) traffic through the OpenRouter gateway would run on a single [Nvidia](https://theinference.org/markets/companies/nvidia) rack worth $6 million to $7 million, by OpenCode's estimate [1]. *Chart: **Closed frontier scores 58 to 59; the best weights you can download sit at 32** Column chart of Intelligence Index v5 scores: closed models Gemini 4 Carbon, Claude Opus 5.5 and GPT-6 Astra at 59, 58 and 57; Kimi K3.1 Pro at 46; Claude Haiku 5.5 at 36; released-weight models DeepSeek V4.1 Flash and GLM-5.3 at 32 each.* *Artificial Analysis Intelligence Index v5 points, 0 to 100, launch snapshot October 27 to 28, 2026. Gemini 4 Carbon has no public API; Kimi K3.1 Pro weights are scheduled for release on November 4. Sources: Artificial Analysis leaderboard, accessed November 3, 2026; Tech Times. [3][5]* ## The free asset is growing a price tag The second hit is contractual. [Alibaba](https://theinference.org/markets/companies/alibaba) Cloud's chief technology officer told the Apsara conference on October 29 that the Qwen 4 family launches "next week"; by November 3, no weights, license or price had appeared [6]. The direction is already on the record. In August, Alibaba made its Qwen3.8-Max weights free to download but required a paid commercial license from firms above $50 million in annual revenue running them as a service, and Moonshot set a $20 million bar on Kimi K3 [7]. A report citing two anonymous people [says Alibaba will seek a revenue share from big commercial users](https://www.businessinsider.com/chinese-ai-models-rankings-artificial-analysis-private-tests-anthropic-openai-2026-11) of its next open model, unconfirmed by the company [6]. Chairman [Joseph Tsai](https://theinference.org/markets/people)'s 470 open models and 4 billion downloads [6], an installed base that now includes [Apple's Siri in China](https://theinference.org/article/apple-puts-alibaba-s-qwen-inside-siri-in-china-in-december-with-no-price-or-revenue-split-public), never budgeted for a license. ## China's checkout counter runs the other way Inside China, prices are falling. Livestreamer Li Jiaqi [sold 100,000 QwenWork memberships](https://www.globaltimes.cn/page/202611/1373520.shtml), about 2 trillion tokens, in half an hour at 0.01 yuan each on Monday night [8], and Taobao's token sales jumped 450 percent month on month on October 20, Double 11's opening day [8]. [Baidu](https://theinference.org/markets/companies/baidu) cut its search API to a promotional 4.9 yuan per 1,000 calls, just under DeepSeek's 5 [9]. *Chart: **Baidu's promo price sits 88 percent under its own list and just below DeepSeek** Bar chart of search API prices per 1,000 calls in yuan: Baidu flagship list 72, enhanced list 40, standard list 36, flagship promo 9.9, DeepSeek 5, Baidu enhanced promo 4.9.* *List and promotional prices per 1,000 calls of Baidu's Search AI API versus DeepSeek's search API, in yuan. Baidu's promo runs October 30 to December 31, 2026, capped at one million calls per buyer; the flagship bundle requires an invitation code. Source: International Search Insights, November 2, 2026. [9]* The other side deserves weight. Kimi still tops Terminal-Bench 4.0, one of the nine v5 tests, at 67 percent, ahead of every closed model [5]. Nathan Lambert, an open-weights researcher at Trillium Labs, calls the old index flattering but the new 12-point gap an overstatement, and expects it to narrow within months [1][5]. Anthropic's own squeeze runs both ways: [third-quarter revenue near $17 billion grew 47 percent sequentially](https://www.msn.com/en-us/news/other/anthropic-just-released-updated-numbers-ahead-of-its-nov-ipo-revenue-growth-slowed-sharply/ar-AA2gCSuz), down from 144, with price cuts costing about $600 million [11]. ## Three dates settle it Kimi's weights land on November 4, Alibaba's "next week" expires on the 8th, and [Anthropic's roadshow can open on the 10th, with trading on Nasdaq under ticker ANT](https://www.msn.com/en-us/news/other/when-is-the-anthropic-ipo-the-stock-could-start-trading-as-soon-as-nov-16/ar-AA2gfbLG) possible from the 16th [10]. The tell is the license text. Permissive terms mean the free substitute is merely weaker; a revenue threshold means it starts charging rent. ## Takeaway Secret benchmarks tripled the open-versus-closed gap to 26 points for downloadable Chinese models, and Alibaba's promised Qwen 4 release arrives with no license on the table, so the free tiers developers build on may start charging rent within days. ## Sources 1. [Business Insider, "Chinese AI models tumble down rankings after a crucial change to testing", 3 November 2026](https://www.businessinsider.com/chinese-ai-models-rankings-artificial-analysis-private-tests-anthropic-openai-2026-11) 2. [Gizmodo, "The Biggest Benchmark in AI Just Updated, Dealing a Major Blow to a Chinese Contender", 29 October 2026](https://gizmodo.com/the-biggest-benchmark-in-ai-just-updated-dealing-a-major-blow-to-a-chinese-contender-2000832292) 3. [Artificial Analysis, Intelligence Index v5 leaderboard, accessed 3 November 2026](https://artificialanalysis.ai/) 4. [WinBuzzer, "Artificial Analysis Overhauls Its Intelligence Index With Harder, Private Benchmarks", 28 October 2026](https://winbuzzer.com/2026/10/28/artificial-analysis-reweights-ai-rankings-toward-private-tests-xcxwbn/) 5. [Tech Times, "Agentic AI benchmarks widen proprietary-model edge to 12 points in artificial analysis v5 overhaul", 29 October 2026](https://www.msn.com/en-us/news/other/agentic-ai-benchmarks-widen-proprietary-model-edge-to-12-points-in-artificial-analysis-v5-overhaul/ar-AA2gkBJk) 6. [shattered.io, "Qwen4 Launch: Alibaba Open-Source Promise in Doubt", 3 November 2026](https://shattered.io/qwen4-launch-alibaba-open-source-promise-doubt-470-models-2026/) 7. [South China Morning Post, "Alibaba adds commercial restrictions to open-weight Qwen3.8-Max AI model", 13 August 2026](https://www.msn.com/en-xl/technology/artificial-intelligence/alibaba-adds-commercial-restrictions-to-open-weight-qwen3-8-max-ai-model/ar-AA2a2He7) 8. [Global Times, "A top Chinese commercial livestreamer sells 2 trillion AI units in 30 minutes", 3 November 2026](https://www.globaltimes.cn/page/202611/1373520.shtml) 9. [International Search Insights, "Baidu Cuts Its Search AI API to ¥4.9 per 1,000 Calls, Undercutting DeepSeek", 2 November 2026](https://readisi.com/baidu-products/baidu-search-ai-api-4-9-yuan-per-1000-calls-undercuts-deepseek-baidu-app-recursive-self-evolution/) 10. [Yahoo Finance, "When is the Anthropic IPO? The stock could start trading as soon as Nov. 16", 27 October 2026](https://www.msn.com/en-us/news/other/when-is-the-anthropic-ipo-the-stock-could-start-trading-as-soon-as-nov-16/ar-AA2gfbLG) 11. [The Motley Fool, "Anthropic just released updated numbers ahead of its Nov. IPO. Revenue growth slowed sharply", 2 November 2026](https://www.msn.com/en-us/news/other/anthropic-just-released-updated-numbers-ahead-of-its-nov-ipo-revenue-growth-slowed-sharply/ar-AA2gCSuz) --- China's free AI models trail Claude by 26 points on secret tests; Alibaba's Qwen may not stay free — The Inference. Canonical: https://theinference.org/article/china-s-free-ai-models-trail-claude-by-26-points-on-secret-tests-alibaba-s-qwen-may-not-stay-free