LiteLLM's '7x faster' claim is missing from its own benchmark data; Stripe pays $8 billion for OpenRouter
A line-by-line check of LiteLLM's own benchmark files finds no 7x gap, while Google sells routing decisions at $0.10 per million tokens and Stripe pays $8 billion for a router. The moat is governance, and it is bleeding.
Vincent Jiangtwo-minute read
The headline, checked against the file it came from
LiteLLM, the open-source AI gateway behind an estimated 92 million PyPI downloads a month 4, sells its Rust rewrite on speed: 0.66 ms of added p99 latency, roughly 22 MB of memory, "4.5x more requests per dollar" than Bifrost 4. A third-party check published on November 11 went back to the raw CSV in LiteLLM's own ai-gateway-bench repository and found the "7x" headline traces to no row of the latency sweep 1. The 4.5 ms Bifrost figure used in the marketing simply does not appear in the data 1.
Bifrost wins at 1 concurrent request and at 64
The committed CSV, checked directly this week, confirms it 3. At one concurrent request Bifrost adds 0.75 ms and Rust 0.89 ms: Bifrost wins 3. At 64 concurrent requests Bifrost adds 28.68 ms and Rust 40.28 ms: Bifrost wins again 3. Rust leads only at the middle points, by as little as 0.02 ms 3.
At the top tested load the two gateways sit 0.1% apart on throughput, 2,722 versus 2,719 requests a second 3. The dollar figure is a model built on assumed cloud prices, not a measured bill 2. The Rust port itself is 3.11% of the repository by source bytes, with 24% of major APIs built 1.
No 7x gap: Bifrost beats LiteLLM Rust at both ends of the sweep
- LiteLLM Rust
- Bifrost
Data
| LiteLLM Rust | Bifrost | |
|---|---|---|
| 1 | 0.89 ms | 0.75 ms |
| 4 | 0.61 ms | 0.63 ms |
| 16 | 0.33 ms | 0.89 ms |
| 64 | 40.28 ms | 28.68 ms |
Google prices the routing layer from above
The same week, Google began selling routing decisions directly: Gemini Decision 1 scores up to 1,000 closed-set questions per pass at $0.10 per million input tokens, zero output tokens billed, and names LiteLLM among its rolling integrations 5. Stripe, meanwhile, agreed to pay $8 billion for OpenRouter, a routing business whose own measured overhead in LiteLLM's benchmark was 61 ms at p50 26. Eight billion dollars for 61 ms says buyers are pricing governance and margin, not milliseconds 26.
The moat is governance, and it is bleeding
What is hard to copy in LiteLLM is not sub-millisecond forwarding; it is the price list, budgets and audit trail. That is also its attack surface: the March PyPI compromise exfiltrated 153 GB across 433,909 files and 2,488 organizations 7, and October's malicious 1.107.0 release came from a deleted-but-unrevoked admin key that stayed valid for four months 7. Watch Bifrost's promised Part 2 rerun 1, and whether the November full-feature lane, where Rust does lead 2, survives outside replication.
Get Platforms by email, free
The cloud, data and developer software AI is built on. In our daily email, every weekday.
