Platforms

Cloudflare now blocks AI agents on any page that shows an ad

Shopify opened its checkout to AI agents this week; Cloudflare closed the ad-funded page instead. Google, Cloudflare and Microsoft are each building the meter that sets what a page of the open web costs an AI.

Vincent JiangVincent Jiangthree-minute read
Share
A wall of colorful lava lamps behind a glass panel bearing the orange Cloudflare logo at a Cloudflare office
1 / 6Slide 1 of 6
The lava lamp wall at a Cloudflare office, a display the company has long used as a source of randomness for its encryption. Cloudflare now blocks AI agents by default on any page that shows an ad.

The default flipped on 15 September

Since 15 September, Cloudflare's−4.68% — Cloudflare, down 4.68 percent today defaults for ad-supported domains let search crawlers through, refuse AI training, and block user-directed agents on any page that shows an ad. Chat-fetch bots and browser-use agents fall under it; there is no opt-out directive for agents yet, only a block 12. Cloudflare set that deadline in July, pressing mixed-use crawlers to split search from training 23.

The same week, Amazon barred Meta's Muse shopping agent from its store over credential risks, and Shopify+0.28% — Shopify, up 0.28 percent today wired Muse into Shop Pay checkout across its merchants: one front of the agent war opened, the content front closed 4.

AI training is now 52% of crawler traffic

AI training alone is 52% of crawler traffic on Cloudflare's network, up from 22% a year earlier, and CEO Matthew Prince calls most traffic on the internet non-human now 13. Site owners are drawing lines: 17% restrict training, fewer than 1% block search 5.

Mixed-use crawlers, built for search and training at once, carry another 36.6% of verified crawler traffic 6. The money moves unseen: 132 million Americans have shopped with AI's help, but only 23% of merchants can see that traffic or its sales, the same agent traffic ad pages now block by default 1.

Three meters, one page, no price

Google's invitation-only pilot pays when content "contributes significantly" to an answer in Gemini, AI Overviews or AI Mode, and Google alone judges what counts 57. Participants get a monthly figure in Search Console with no arithmetic behind it 8. One executive briefed on the program calls it "quite black box"; another said the offers were too low to opt in 5.

Cloudflare's Pay Per Crawl is mechanical: sites set the price from $0.001 a fetch, and a paying crawler gets the page, HTTP 200 and all 5. Microsoft−1.02% — Microsoft, down 1.02 percent today licenses content to Copilot on "delivered value" it has not defined publicly 5. The payers are AI labs and agent builders, in cash at the meter or in lost inventory when their agents are blocked. Whichever meter wins will set the price of the open web for AI.

The toll collector's books

Cloudflare sells both the blockade and the meter. Revenue has risen ten straight quarters, to $696.1 million in Q2 2026, up 35.9% on the year, and the quarter still lost $170 million 96. The market has paid up anyway: $115.2 billion of market cap after a 43.9% run in 90 days 6.

Cloudflare revenue has risen ten straight quarters while still losing money

$300M$400M$500M$600M$700MQ4 '23Q2 '24Q4 '24Q2 '25Q4 '25Q2 '26$696.06MQuarterly net loss: $170M
Data
Revenue
Q4 '23$362.47M
Q1 '24$378.6M
Q2 '24$401M
Q3 '24$430.08M
Q4 '24$459.95M
Q1 '25$479.09M
Q2 '25$512.32M
Q3 '25$562.03M
Q4 '25$614.51M
Q1 '26$639.76M
Q2 '26$696.06M
Quarterly revenue in USD millions as reported in Cloudflare's SEC filings via Sharadar, Q4 2023 through Q2 2026; annotation notes the Q2 2026 net loss from the same filings.9

Google's exit ramp, and what to watch

Google points to Google-Extended, a robots.txt token it says never touches Search ranking, and calls the pilot a learning phase 3510. That token opts out of Gemini training only; a separate Search Console setting governs AI Overviews and AI Mode 10.

Cloudflare's framing still holds: an ad marks a page built for a person 1. The exposed party is the small ad-funded publisher, because an agent reads, buys nothing, and never sees the ad.

The next readings are dated: Google's URL-level Google-Extended tools land within weeks, Cloudflare's single control over how much content appears in AI summaries arrives by early next year, and Microsoft's Bing no-training support is targeted for early 2027 510. The storefronts opened their checkouts to agents; the content layer just installed a toll booth.

Get Platforms by email, free

The cloud, data and developer software AI is built on. In our daily email, every weekday.

Loading NET

Reader comments

0 comments

    More in Platforms

    All in this topic →

    Platforms

    Pump.co's claimed spend under management jumped from $600 million to $10 billion in six weeks

    The Y Combinator-backed startup charges customers $0, says cloud providers pay it, and is hiring account executives and partnership leads anyway. For FinOps rivals that bill monthly, and for anyone underwriting the next round, the homepage's new $10B+ figure is the first claim to test.

    Platforms

    LangChain raises $300 million at $3 billion, and the cap table now includes its own customers

    LangChain's agent-governance layer just repriced 2.4 times in 13 months on a usage number its own lead investor calls apples to oranges, and two of its new backers sell competing agent controls. Whose money moves: anyone pricing AI application dependencies against what the model labs will bundle next.

    Platforms

    Parcha's Greptile bill passed $7,000 a month, so it built an 18-cent reviewer and open-sourced it

    The bill is the first customer-published measure of what usage-priced AI code review costs when agents write the pull requests, and it landed the week Cohere bought the reviewer ranked above Greptile on an independent leaderboard. The money at stake is Greptile's $750 million mark and every seat-priced review vendor's renewal.

    Platforms

    DXC booked 80 cents of new orders for every dollar of revenue last quarter

    DXC spent last week selling OASIS, the agent platform it says cuts incident-resolution time 95% inside banks and hospitals. Its September order book, 80 cents of new work per dollar billed, says the money has not followed the demos, and the core it must replace is still running off.

    Platforms

    Palantir's Karp says a wealth tax 'is going to need surveillance' as the stock sits at 219 times earnings

    Recorded on November 10 for Walter Isaacson's podcast and carried to more than 14,000 Reddit points, the interview landed in a week that took 17.55 percent off Palantir. At 219 times earnings, holders own the founder's story as much as his contract book.

    Platforms

    Amtrak's booking site failed during the FAA's four-hour national ground stop

    Amtrak blamed a "systemwide update", the same failure mode that grounded the country's flights: a change pushed into a system that is never allowed to stop. The money Congress is asked to add will buy change discipline on live systems, and the vendors selling staged rollouts and tested failover are the layer positioned to be paid for uptime.

    Subscribe to TheInference for free