China's free AI models trail Claude by 26 points on secret tests; Alibaba's Qwen may not stay free
Artificial Analysis moved 60 percent of its benchmark onto secret tests, and the best downloadable Chinese models now trail Anthropic's Claude by 26 points. Alibaba's Qwen 4, promised for this week, still has no license on the table, just as Anthropic readies a Nasdaq listing at $17 billion a quarter.
Vincent Jiang · 3 min read
The leaderboard went private, and the gap quadrupled
Artificial Analysis, the independent firm whose index is AI's most-cited scorecard, rebuilt its exam on October 27 14. Sixty percent of the score now rests on secret, held-out questions, up from 45 percent, so labs can no longer train to the test 14. The gap between the best open-weight model and the leader widened overnight from 3 points to 12 1. Moonshot AI's Kimi K3.1 Pro, fourth in the world on the old index, fell to ninth 2. It scores 46, but its weights are not downloadable until November 4 35.
Among weights a developer can actually download, the fall is steeper. DeepSeek's V4.1 Flash and Zhipu's GLM-5.3 lead at 32 points, 26 behind Anthropic's$350B — Anthropic, private, latest valuation $350B Claude Opus 5.5 at 58 34. Claude Haiku 5.5, Anthropic's smallest model, scores 36 and beats every downloadable release 5. A full day of DeepSeek traffic through the OpenRouter gateway would run on a single Nvidia+3.08% — Nvidia, up 3.08 percent today rack worth $6 million to $7 million, by OpenCode's estimate 1.
Closed frontier scores 58 to 59; the best weights you can download sit at 32
Data
| Intelligence Index v5 | |
|---|---|
| Gemini 4 Carbon | 59 |
| Claude Opus 5.5 | 58 |
| GPT-6 Astra | 57 |
| Kimi K3.1 Pro | 46 |
| Claude Haiku 5.5 | 36 |
| DeepSeek V4.1 Flash | 32 |
| GLM-5.3 | 32 |
The free asset is growing a price tag
The second hit is contractual. Alibaba−0.97% — Alibaba, down 0.97 percent today Cloud's chief technology officer told the Apsara conference on October 29 that the Qwen 4 family launches "next week"; by November 3, no weights, license or price had appeared 6. The direction is already on the record.
In August, Alibaba made its Qwen3.8-Max weights free to download but required a paid commercial license from firms above $50 million in annual revenue running them as a service, and Moonshot set a $20 million bar on Kimi K3 7. A report citing two anonymous people says Alibaba will seek a revenue share from big commercial users of its next open model, unconfirmed by the company 6.
Chairman Joseph Tsai's 470 open models and 4 billion downloads 6, an installed base that now includes Apple's Siri in China, never budgeted for a license.
China's checkout counter runs the other way
Inside China, prices are falling. Livestreamer Li Jiaqi sold 100,000 QwenWork memberships, about 2 trillion tokens, in half an hour at 0.01 yuan each on Monday night 8, and Taobao's token sales jumped 450 percent month on month on October 20, Double 11's opening day 8. Baidu+0.39% — Baidu, up 0.39 percent today cut its search API to a promotional 4.9 yuan per 1,000 calls, just under DeepSeek's 5 9.
Baidu's promo price sits 88 percent under its own list and just below DeepSeek
Data
| Value | |
|---|---|
| Baidu Flagship (list) | ¥72/1k calls |
| Baidu Enhanced (list) | ¥40/1k calls |
| Baidu Standard (list) | ¥36/1k calls |
| Baidu Flagship (promo) | ¥9.9/1k calls |
| DeepSeek Search | ¥5/1k calls |
| Baidu Enhanced (promo) | ¥4.9/1k calls |
The other side deserves weight. Kimi still tops Terminal-Bench 4.0, one of the nine v5 tests, at 67 percent, ahead of every closed model 5. Nathan Lambert, an open-weights researcher at Trillium Labs, calls the old index flattering but the new 12-point gap an overstatement, and expects it to narrow within months 15. Anthropic's own squeeze runs both ways: third-quarter revenue near $17 billion grew 47 percent sequentially, down from 144, with price cuts costing about $600 million 11.
Three dates settle it
Kimi's weights land on November 4, Alibaba's "next week" expires on the 8th, and Anthropic's roadshow can open on the 10th, with trading on Nasdaq under ticker ANT possible from the 16th 10. The tell is the license text. Permissive terms mean the free substitute is merely weaker; a revenue threshold means it starts charging rent.
Get Model by email, free
The AI labs, their models, and the business behind them. In our daily email, every weekday.
