A model has no single price per token

The rate a token is billed at is not a property of the model. It is a property of the request that carried it: how large the whole prompt was, which kind of token it was, and which deployment served it. Three separate projects shipped cost-pricing fixes in the past week, each of them repairing the same assumption — that a model has one input price and one output price — and each of them wrong in the same place: on the largest and least typical requests.

Where a calculator and an invoice diverge

An AI token cost calculator multiplies token counts by one published rate per model. That is right for a short, single-turn request, and it goes wrong on the requests that dominate an agent bill, because at least four things other than the model name select the rate:

• The size of the whole request. Some models price in threshold tiers, so a prompt that crosses a size boundary is billed at a different rate than the same prompt one token smaller.
• Which kind of token it is. Plain input, a cache write, a cache read and reasoning tokens are separate billed dimensions with separate rates, and a tiered model has a rate for each of them per tier.
• Whether a tier is all-or-nothing or graduated. Graduated means only the tokens past the boundary are repriced. All-or-nothing means every token in the request is.
• Which deployment served it. A deployment bought as flat capacity has a correct per-token price of zero, because the capacity is already paid for; a fallback hop to a pay-per-token deployment of the same model does not.

The same defect, three projects, one week

A LiteLLM pull request on 2026-08-13 rebuilt tiered pricing and lists what was mispriced before it:

• Pricing schema rejected cache_creation_input_token_cost inside tiers
• Cache-creation tokens were never billed at tier rates
• Dashscope tiers were billed graduated, not all-or-nothing
• Tiered-only models (volcengine) were billed at $0

Two of those are worth separating. Cache-creation tokens missing from the tier schema is a rate applied to the wrong token class. Graduated where the provider charges all-or-nothing is a rate applied over the wrong span — same table, same tokens, and the error grows with the request.

Four days later, on 2026-08-17, a Helicone pull request fixed the cache-write side of the same problem and states why the two are coupled:

For Anthropic's long-context threshold (> 200k tokens), cache writes count toward the same total-prompt that drives the inputCost threshold.

The PR reports that pricing those writes at the base tier "undercharges them by ~2x". Note the direction: a tool that computes cost from your token counts was reporting less than the provider charged, on the biggest prompts in the set. Cache-write pricing is the same surface as broken prompt caching seen from the accounting side rather than the engineering side.

A Langfuse pull request the same day made pricing conditions match on observation attributes, so that the platform can "match exact top-level model-parameter and metadata values during direct v4 event ingestion". That is the third mechanism: a parameter on the request — not the model — chooses the rate.

And a LiteLLM fix on 2026-08-13 covers the deployment axis, stopping per-token billing on a provisioned-capacity deployment:

• A PTU deployment billed per token on top of its flat capacity cost
• Unset per-token price fell back to the public cost map
• So the double charge was the default, not an opt-in

A fallback to the public price table is the failure mode worth remembering. When a cost pipeline does not know the rate, it does not report "unknown" — it substitutes the list price, and the resulting number looks exactly as authoritative as a correct one.

Agent runs sit at the end of the distribution these errors live in

Every one of these mispricings is small on a short prompt and large on a long one, which is the shape of an agent workload. A run appends tool results and prior turns to its own context, so the last call of a run is the biggest — context regrowth is that mechanism — and it is the last call that crosses a threshold. Fan-out multiplies how many calls arrive in the expensive band at once: our July 2026 scan of public agent repositories found a median of 2 agents per repository and 8 at the 90th percentile (method and limitations).

The consequence for the unit is direct. A single rate per model makes cost per run a linear function of tokens, so doubling the context reads as double the cost. Under an all-or-nothing threshold it can be more than double, and the extra appears in no per-call figure that was computed with the base rate.

Four checks on a cost figure

1. Price each call from its own usage fields, per token class. Cache reads, cache writes, input, output and reasoning tokens are five numbers, not one input total and one output total.
2. Store the rate next to the row. A cost row that records only tokens and dollars cannot be re-derived later, and cannot be audited when the price table changes — the argument in tracking tokens is not the same as tracking cost.
3. Test at the boundary, not at the average. Take one run whose largest call crosses a documented threshold and reconcile that run against the invoice. A pipeline that is accurate on 5k-token calls tells you nothing about a 250k-token one.
4. Ask which deployment served the call. Reserved capacity and a pay-per-token endpoint are two different rates for one model name, and a fallback can move a call between them mid-run.

What this means for a budget

A pre-call check has to estimate, and the rate is one more thing it cannot know in advance: which tier a request lands in depends on its total size, and whether its prefix is billed as a write or a read depends on the provider's cache state. That is the same class of gap as the unknown output length in a budget check cannot price the call it allows, and it is bounded the same way — size the envelope on the worst case, then reconcile the estimate against the usage counts that come back.

How this works in Capsera

Capsera records token counts per call, attributed to the agent that made the call, so a run's cost is a sum over its own calls rather than an average rate applied to a daily total. Budgets are checked in-process before the provider call, and the check reads recorded spend, which means the pre-call figure is an estimate and the response is the measurement — tracking the difference per run is what tells you whether your price model holds at the top of your size distribution. Capsera stores hashes and token counts, never prompt or response content.

If what you need is a provider-breadth cost map maintained across many models and deployment types, that is a gateway's job: LiteLLM is MIT-licensed and free, covers 140+ providers, and is where the tiered-pricing work quoted above happened. For per-call trace inspection, prompt-content debugging or evals, Langfuse is open source, free, and built for it.