Routing swaps the model, not the request

A router substitutes the model. It does not substitute the request. Every model-relative field your client sent — the output ceiling, the model id, the assumption that one request produces one billed call — was sized for the model you configured, and is now being interpreted by a model you did not choose. The cost of that mismatch does not appear as a higher price per token. It appears as a second attempt on the same run.

The accounting side of routing — which baseline a savings figure was priced against, and what a cold per-model cache costs on a switch — is a separate problem covered elsewhere. This is about the request itself.

The output ceiling was sized for a different model

A pull request opened on 2026-09-08, resolving max_tokens to the tier model's ceiling on auto-routed requests, states the problem as one client value meeting many models:

A client behind an auto-router sends one max_tokens for every tier

and then names both failure directions:

Sized for Haiku, it starves Sonnet/Opus thinking and returns empty answers

Sized for Sonnet, Haiku rejects it and the fallback serves Sonnet instead

Read those as cost events rather than as bugs. An empty answer is a billed call that produced nothing usable, and whatever issues the request will ask again. A rejection that triggers a fallback ends with the expensive model serving the request anyway — the run pays the routing hop and then pays the tier it was being routed away from. In both cases the per-call price went down and the number of calls behind one finished task went up.

The proposed fix is worth reading for what it implies. It sets max_tokens after the tier is picked, and takes the value from "the deployment's model_info.max_output_tokens, else the cost map". That makes the shape of your request a function of a metadata table. A separate pull request three days earlier, adding a cost map sync bot, describes the state of that table: "OpenRouter and Vercel AI Gateway ship new models weekly; the map lags by weeks", "Stale entries carry old prices, context windows, and capability flags", and "The weekly LLM-based updater has failed every run since August 2". Both are proposals in an open repository rather than shipped behaviour. The shape they share is the point: once a router rewrites model-relative parameters, a stale row in a catalogue is a live change to what your requests ask for.

One client request, more than one billed call

A pull request opened on 2026-09-07, adding a shunt option to the auto router, moves work rather than repricing it:

Large file reads burn expensive-model tokens on bytes nobody needs

Bounds big reads, delegates them to a cheap model

Delegation is not substitution. The expensive model still runs the task; a cheap worker runs the read underneath it. That is a second billed unit behind one client request, and the PR says where it lands: "Worker spend still bills to the calling key". Billing it to the caller is the right instinct — the key is the only identity a proxy reliably has. It is also the coarsest one available, which is the whole difficulty with the identity a limit is keyed to: one key covers every agent in a process, so a per-key total cannot say which agent's read was shunted or which run absorbed the worker call.

This is what people are already doing by hand. A Hacker News comment on 2026-09-05, under a thread about a new model on OpenRouter, describes a skill written to "delegate the use of bulk reading of docs/code and implementation to a sub-agent". Another on 2026-09-09, in a thread about a cheaper flash model, puts the same division of labour more bluntly: "I load up planning and tasks in Opus or Sol, and just have glm flash workers go to town every night." Both descriptions are of one task fanning into calls at several price points. The count of those calls, per finished task, is the number that moves.

The client issues model ids you never configured

A pull request opened on 2026-09-08, adding lite configure claude, reports a third way the request stops matching the configuration:

Claude Code requests its own model ids for sub-agents and background helpers, which 400 on a proxy that does not serve them

A rejected request spends nothing at the provider, so this one is cheap to discover and easy to miss. What it establishes is the inventory problem: the set of model ids arriving at your routing layer is a superset of the set your application names, because a coding harness spawns sub-agents and background helpers with model choices of its own. Those are calls you never wrote, and a routing policy written against the models in your config does not describe them.

A fourth pull request, opened on 2026-09-09 to surface the routed model and session spend in Claude Code and Codex, is the same gap read from the other end: "Nothing tells a developer what a session actually cost versus the router's baseline." The unit it reaches for is the session, not the call, because the call-grained view is where all of this disappears.

What to record

1. Billed calls per completed run, before and after a routing rule. A rule that lowers the price of a call and adds a call has not saved anything. Cost per run is the only unit in which a substitution and its consequences land on the same line.
2. Empty and truncated responses as their own outcome. Not a success, not an error — a distinct terminal state with a cost attached. If they fold into either bucket, the retry they cause is attributed to the wrong thing; this is the same denominator question a retry raises.
3. The tier that served each attempt, and the ceiling that was applied to it. Once parameters are resolved per tier, "which model answered" is no longer enough to reproduce the request.
4. Fallback-served requests, counted separately. A request that was routed cheap, rejected, and then served by the expensive model is the most expensive path through the system and it looks like an ordinary call on the expensive tier.
5. An identity that survives delegation, so a worker call issued on your behalf lands on the agent and run that caused it rather than on the credential that happened to pay.

How this works in Capsera

Capsera runs in-process. capsera.init() patches the provider clients — Anthropic messages, OpenAI chat completions, OpenAI embeddings and Google Gemini, sync and async — so every request your process makes is captured with the parameters it actually carried and the usage the provider actually reported, without a call site changing. @capsera.agent(name="reader") adds who, not whether: because the agent stack is a ContextVar rather than thread-local state, a sub-agent inherits its parent's identity and a delegated call made inside that scope reports as itself, under the run that caused it. So the counts above are readable per agent and per run rather than per key.

Capsera's own routing engine is SDK-side and pre-call: cost cap, task type, model map, then fallback chain, first match wins, cross-provider routing blocked, and fail-open if rules cannot be fetched. Budgets are checked before the provider call and scoped per agent, per team or globally, so an agent that starts fanning one task into four attempts is refused at its ceiling rather than reconciled afterwards — the ceiling holds in dollars whichever tier ends up serving the call.

The limit is honest and worth stating. If the substitution happens inside a gateway, the rewritten request is the gateway's, in the gateway's process, and Capsera sees the one request your code sent. Those rows live in the gateway's own spend log. LiteLLM is MIT-licensed, free, covers 140+ providers, and enforces hard budgets per key, team, organisation and model, which is where to bound a router it owns; we auto-detect a LiteLLM proxy rather than replace it. And if the question is why a cheap tier returned an empty answer — the content of the exchange rather than its cost — Langfuse is open source, free, and built for that. Capsera stores hashes and token counts, never prompt or response content.