--- title: "Oracle and AWS shipped LiteLLM gateway blueprints on Nov. 12, with budgets that don't enforce by default" description: "The fine print in both cloud blueprints shows the routing layer does not meter spend by default and does not route consistently, while OpenAI's subscription meter expands to 11 outside apps. Anthropic prices its IPO on November 18, and revenue per agent task is part of that argument." publisher: "The Inference" section: "Platforms" published: 2026-11-13T20:03:53.678Z modified: 2026-11-13T20:03:53.678Z canonical: https://theinference.org/article/oracle-and-aws-shipped-litellm-gateway-blueprints-on-nov-12-with-budgets-that-don-t-enforce-by-default language: en keywords: "LiteLLM, AI Infrastructure, OpenAI, Anthropic, Enterprise AI" --- # Oracle and AWS shipped LiteLLM gateway blueprints on Nov. 12, with budgets that don't enforce by default > The fine print in both cloud blueprints shows the routing layer does not meter spend by default and does not route consistently, while OpenAI's subscription meter expands to 11 outside apps. Anthropic prices its IPO on November 18, and revenue per agent task is part of that argument. [Oracle](https://theinference.org/markets/companies/oracle) and [AWS](https://theinference.org/markets/companies/amazon) both published production blueprints on November 12 that put [LiteLLM's open-source AI gateway](https://theinference.org/markets/companies/litellm) between enterprise apps and the models behind them [1][2]. Read the fine print, and the layer they are handing their customers does not yet meter the money. [AWS's own guide](https://aws.amazon.com/blogs/machine-learning/building-a-self-service-llm-gateway-with-amazon-bedrock-and-litellm/) warns that with Bedrock inference-profile ARNs, LiteLLM records spend as $0, so per-key budgets "NEVER enforce" until an operator sets per-token prices by hand [2]. The self-service key portal that pattern needs is a paid Enterprise feature; the open-source gateway is free for up to five users [2]. [Oracle's reference](https://blogs.oracle.com/ai-and-datascience/litellm-routes-oci-generative-ai-models) ships a router trained on 3,590 graded answers across 20 OCI-hosted models in eight routes, but when every answer was labeled three times, the same cheapest passing route was chosen for only 66% of requests [1]. ## Routing saves money, and that deflates vendor revenue The router's economics run both ways. It sends a simple question to a model that costs about a thousandth of a dollar, while a four-turn code fix lands on a coding model that costs about $0.07, some twenty times as much [1]. That is routing working as designed, and it is revenue deflation for model vendors if enterprises adopt it. The chart below shows the same trade-off on Oracle's held-out set: routing policies that always buy quality pay several times what the trained router pays, and the router stays close to the cheap end [1]. *Chart: **Always-reasoning routing costs nearly nine times always-fast on Oracle's held-out set** Column chart of the cost of Oracle's 73 graded answers under five routing policies: always fast models $0.043, LiteLLM's heuristic $0.082, the trained router $0.306, LiteLLM's LLM classifier $0.338, always reasoning models $0.376.* *Cost of answering Oracle's 73-question held-out set under five routing policies, in US dollars. The trained router sits near the cheap end while staying close to always-reasoning quality. Source: Oracle Blogs, 12 November 2026. [1]* ## OpenAI moved the meter inside its own subscription [OpenAI](https://theinference.org/markets/companies/openai) answered on November 10 by becoming its own meter: ChatGPT Business allowances now pay for AI inside 11 outside apps through Sign in with ChatGPT, with [Lovable](https://theinference.org/markets/companies/lovable) to follow; Enterprise and Edu are not eligible [3]. Metering inside the subscription, outside the gateway, is the vendors' hedge against exactly this deflation. [Anthropic](https://theinference.org/markets/companies/anthropic) [prices its IPO](https://theinference.org/article/anthropic-files-for-ipo-at-70-to-78-a-share-near-1-trillion-with-amazon-s-239-billion-mark-on-the-line) on November 18, a $70 to $78 range implying about a $1.3 trillion valuation [7]; what an agent task is worth after routing is part of the revenue argument that price will be argued on. [Mercor](https://theinference.org/markets/companies/mercor)'s new $20 billion round funds evaluation and post-training, the business of measuring whether models complete tasks, and the startup was itself bitten by LiteLLM in April's hack [6]. The market is already pricing the measurement layer. ## The free layer carries a breach history The layer both clouds handed out has a record we covered here in October, when [Check Point](https://theinference.org/markets/companies/check-point-software) found LiteLLM let any signed-in user seize admin ([that piece](https://theinference.org/article/check-point-found-litellm-let-any-signed-in-user-seize-admin-97-million-downloads-a-month), and [our benchmark/OpenRouter piece](https://theinference.org/article/litellm-s-7x-faster-claim-is-missing-from-its-own-benchmark-data-stripe-pays-8-billion-for-openrouter) from November 12). What is new since: an admin key exposed in May sat unrevoked for over four months before an attacker used it to hijack the release pipeline, and [the malicious version 1.107.0 shipped on PyPI and official Docker tags](https://news.jkn.co.kr/editions/english/post/1076744) for about two hours on October 21 [4][5]. [March's earlier compromise produced 118,829 runner dumps from 2,488 organizations in a 153GB archive](https://blog.gitguardian.com/inside-the-litellm-hack/), many with live credentials nobody can attribute [4]. The consequence for adopters: revocation, not deletion, is what closes a leaked key, and per-token prices set explicitly are what make budgets enforce at all. ## The next contract decides whose meter counts The chain ends at whose money moves. Oracle and Amazon win either way: they sell the capacity the gateway routes, and the routing saves the customer money [1][2]. The contested link is the meter. Enterprises need capped, attributed spend; vendors need usage they meter themselves. The concrete test: whether the next enterprise inference contract prices per routed model call, inside a gateway the customer controls, or rides a subscription allowance the vendor meters. The first deflates revenue per agent task; the second protects it. Anthropic's IPO price, set on November 18, will be argued partly on that question [7]. ## Takeaway Both clouds' LiteLLM blueprints leave spend metering to the operator: Bedrock spend records as $0 until per-token prices are set by hand, and Oracle's router picks the same cheapest passing route for only 66% of requests. ## Sources 1. [Oracle Blogs, LiteLLM now routes every request to the right OCI Generative AI model with a trained classifier, 12 November 2026](https://blogs.oracle.com/ai-and-datascience/litellm-routes-oci-generative-ai-models) 2. [AWS Machine Learning Blog, Building a self-service LLM gateway with Amazon Bedrock and LiteLLM, 12 November 2026](https://aws.amazon.com/blogs/machine-learning/building-a-self-service-llm-gateway-with-amazon-bedrock-and-litellm/) 3. [ORIENS, OpenAI lets ChatGPT Business subscriptions work in outside tools via Sign in with ChatGPT, 12 November 2026](https://ai-oriens.com/en/news/sign-in-with-chatgpt-business/) 4. [GitGuardian Blog, Inside the LiteLLM hack: 153GB, 433,909 files, 2,488 organizations, updated 3 November 2026](https://blog.gitguardian.com/inside-the-litellm-hack/) 5. [JKN, Official channels compromised: AI tool LiteLLM used to distribute malware, 7 November 2026](https://news.jkn.co.kr/editions/english/post/1076744) 6. [Forbes, The World's Youngest Self-Made Billionaires Just Doubled Their Fortunes, 10 November 2026](https://www.forbes.com/sites/aliciapark/2026/11/10/the-worlds-youngest-self-made-billionaires-just-doubled-their-fortunes/) 7. [Forbes, Anthropic IPO Targets $1.3 Trillion Valuation And A $90 Billion Raise, 10 November 2026](https://www.forbes.com/sites/gabrielalinzainescu/2026/11/10/anthropic-ipo-targets-13-trillion-valuation-904-billion-raise/) --- Oracle and AWS shipped LiteLLM gateway blueprints on Nov. 12, with budgets that don't enforce by default — The Inference. Canonical: https://theinference.org/article/oracle-and-aws-shipped-litellm-gateway-blueprints-on-nov-12-with-budgets-that-don-t-enforce-by-default