LLM Gateway
The LLM gateway built for cost attribution
Route OpenAI, Anthropic, Gemini and 16+ providers through one base URL, and get every token attributed to a team, feature, environment or customer. Most gateways were built to solve routing. This one was built to answer what your AI actually costs, and who spent it.
What is an LLM gateway?
An LLM gateway is a single endpoint that sits between your applications and the model providers they call. Rather than every service holding its own provider keys and talking directly to OpenAI, Anthropic or Google, they all call the gateway, which forwards each request to the right provider and returns the response unchanged. You will also see it called an AI gateway: same pattern, slightly broader name.
Because every call passes through one place, the gateway is the only component that sees all of your AI traffic. That makes it the natural control point for three things: measuring what was spent, attributing it to whoever caused it, and deciding what is allowed to run at all.
Most gateways solve routing. Few solve cost.
The first generation of LLM gateways existed to keep traffic flowing: failover when a provider degrades, load balancing across keys, caching repeated calls. All genuinely useful, and all orthogonal to the question finance is asking.
What a routing-first gateway tells you
Which API key spent what, this month, in total. Useful for an engineer chasing a spike. Not enough to tell a CFO why gross margin moved, or to apportion cost to a cost centre.
What a cost-first gateway tells you
What this feature cost per customer last month, split by model, with cached and reasoning tokens priced separately and the total reconciling against the provider invoice.
What the gateway gives you
Everything finance and engineering need to see, attribute and control LLM API costs, captured at the point every request passes through.
One base URL, 16+ providers
Point your existing OpenAI, Anthropic or other official SDK at the gateway. One base URL change, no wrapper library, no rewrite of your call sites.
Attribution built into the routing layer
Tag requests with team, feature, environment or customer as they pass through. Because the gateway sees every call, attribution is complete rather than sampled.
Token-level cost, priced per model
Prompt, completion, cached and reasoning tokens captured separately and priced against each provider's own rate card, so the dashboard reconciles with the invoice.
Budgets with burn alerts
Set monthly budgets per team or cost centre and get alerted at configurable burn thresholds, while there is still time to act rather than at month end.
Model and provider allow-lists
Decide which models and providers each API key may call. Unapproved or expensive models are blocked at the gateway, before the request reaches the provider.
Prompts are never stored
The gateway extracts usage metadata only. No prompt or response content is retained, so there is no corpus to defend in a security review.
A gateway you do not have to run
Self-hosting a gateway means a service to deploy, a database to keep and someone on call for both. This one is managed.
No gateway containers to deploy, no database to back up, no cache to scale. The gateway is a managed service.
Requests are served from the location nearest your users, without you provisioning a single region.
Usage and cost capture run asynchronously after the response is sent, so the gateway stays off your critical path.
AI SpendOps vs a routing-first LLM gateway
These are different tools for different problems, and plenty of teams run both. Here is the honest split.
| AI SpendOps | Routing-first gateway | |
|---|---|---|
| Primary job | Cost attribution and governance | Routing, failover and throughput |
| Drop-in integration (base URL change) | ||
| Providers behind one endpoint | 16+ | Often 100+ |
| Routing logic (fallbacks, load balancing, caching) | Pass-through, no rewriting | Extensive |
| Per-request token capture | ||
| Cache, reasoning and modality tokens split out | Partial, varies by provider | |
| Attribution by team, feature, environment, customer | Custom dimensions, every plan | Tag-based at best |
| Budgets | Tracked, with burn alerts | Varies |
| Model and provider allow-lists at call time | Varies | |
| Prompt and response storage | Never stored, metadata only | Usually configurable |
| Infrastructure you run | None | Self-hosted, or managed at a price |
| Reporting audience | Finance and engineering, same numbers | Engineer-facing dashboards |
Live in an afternoon
Keep your SDK, keep your provider keys, change the base URL your client points at. Add headers for the dimensions you want to report on, and the gateway does the rest.
- 1Point your existing OpenAI or Anthropic SDK at the AI SpendOps base URL
- 2Add headers for team, feature, environment or customer on each call
- 3Watch token-level cost, attribution and budget burn appear in the dashboard
LLM gateway questions
What is an LLM gateway?
An LLM gateway is a single endpoint that sits between your applications and the large language model providers they call. Instead of each service talking directly to OpenAI, Anthropic or Google, they all talk to the gateway, which forwards the request to the right provider. Because every call passes through one place, the gateway is the natural point to capture usage, apply policy and attribute cost. Most gateways were built to solve routing problems such as failover and load balancing. AI SpendOps is built around the cost question instead.
What is the difference between an AI gateway and an LLM gateway?
In practice the terms are used interchangeably, and most tools answer to both. AI gateway is the slightly broader label, sometimes covering image, audio and embedding models as well as text. LLM gateway is the narrower term for the same pattern applied to large language model APIs. Whichever name you use, the architecture is identical: one endpoint in front of many providers. AI SpendOps works across text, image, audio and embedding traffic, so either label fits.
Which LLM gateway has cost tracking?
Most gateways log spend per API key, which answers which key spent what. That is a different question from what a feature cost per customer last month. AI SpendOps captures prompt, completion, cached and reasoning tokens separately on every request, prices them against each provider's own rate card, and attributes the result to a team, feature, environment or customer through custom dimensions. The output is a cost breakdown finance can apportion to cost centres, not a spend table engineers have to query.
Can an LLM gateway track spend by team and feature?
Only if it captures the dimension at request time, which is why the gateway is the right place to do it. With AI SpendOps you attach headers for team, feature, environment or customer on each call, and untagged requests can be rejected at the gateway if you want complete coverage. Attribution is then exact rather than reconstructed after the fact from provider billing data.
Does routing through an LLM gateway add latency?
It should not, and with AI SpendOps it does not meaningfully. The gateway runs on Cloudflare's edge across 330+ cities, so requests are served near your users, and all usage, token and cost capture happens asynchronously after the response has already been sent to your application. The measurement work never sits on the response path.
Do I have to replace my OpenAI SDK to use the gateway?
No. AI SpendOps is OpenAI-compatible, so you keep the official SDK you already use and change the base URL it points at. You also keep your own provider keys. There is no wrapper library to adopt, no vendor SDK to learn, and backing the change out is the same one-line edit in reverse.
Which enterprise AI gateway helps finance teams centralise AI billing?
The requirement finance usually has is one number per team, per feature and per cost centre across every provider, reconciled to the provider invoices. A gateway is the only place that can produce it, because it is the only component that sees every request regardless of which team made it. AI SpendOps was designed for that reporting job specifically, with apportionment by cost centre, budgets with burn alerts, and figures that engineering and finance both read from the same source.
Is AI SpendOps an LLM gateway or a cost platform?
Both, and deliberately in that order. The gateway is how the data is captured, the cost platform is what the data is for. It is worth being clear about what it is not: AI SpendOps passes requests through rather than rewriting them, so it does not do fallbacks, load balancing or semantic caching. If routing logic is the problem you are solving, a routing-first gateway is the better tool, and the two can sit together.
Put a gateway in front of your AI spend
Token-level attribution across 16+ providers, through one base URL. First 3 months free.
Sign Up