A budget check cannot price the call it allows

A pre-call budget check knows what an agent has already spent. It does not know what the call it is about to allow will cost, because the token count only arrives with the response. So every hard ceiling is really the limit plus an overshoot — the calls already in flight, plus the one being permitted — and the question worth asking of any spend control is not whether that overshoot exists, but how large it can get and whether anyone has bounded it.

The number that arrives last

A comment on a Hacker News launch thread this week stated the mechanism without hedging:

The cost predictability problem is real and it comes from a single mechanical fact: you cannot know what an LLM call costs before you make it, because the token count only comes back with the response.

Half of that is fixable, and the two halves are worth separating. The input side of a request is fully determined before you send it — the system prompt, the message history and the tool definitions are all sitting in memory — and providers will count it for you. As of 2026-08-16, Anthropic's docs describe a token counting endpoint on the Messages API that returns the input token count for a request without running it (token counting). An Anthropic token counter therefore answers the first half of the question: how many input tokens this call will be billed for, before the call. Nothing answers the second half. Output tokens exist only once the model has generated them, and how many that will be is usually the thing you invoked the model to find out.

A max-token cap is a bound, not an estimate

A comment on a second Hacker News thread this week, under a research agent that advertises a spend budget, asked the follow-up that gets to the mechanism:

how do you handle the LLM spending more than you would expect on a request? Or is this handled by max_tokens and some form of pricing table? (and if so, how does caching play a role?)

On the first part: max_tokens is the only pre-call handle on output cost, and it is a worst case rather than a prediction. A call capped at 4000 output tokens usually returns far fewer, so an estimate built on the typical case is right most of the time and stops being right exactly when behaviour goes wrong — which is the case the ceiling exists for. A ceiling has to be sized on the 4000.

On caching: it widens the input estimate rather than narrowing it. The same prefix is priced differently depending on whether it reads from the provider's cache, is written to it, or misses, and which of the three happened comes back in the usage figures with the response. Counting tokens tells you how many input tokens the request carries. It does not tell you which rate they will be billed at.

Concurrency multiplies the part you cannot price

Between the check and the record of what the call actually cost, everything in flight is unpriced. One agent calling in sequence overshoots by one worst case. A fan-out of eight parallel sub-agents that all clear the check before any of them returns overshoots by eight: with each capped at 4000 output tokens, that is 32000 output tokens billed after the ceiling said yes. Eight is not an invented width — our July 2026 scan of public agent repositories found a median of 2 agents per repository and 8 at the 90th percentile (method and limitations).

Enforcement layers are starting to carry that dimension explicitly. A LiteLLM pull request opened on 2026-08-11 proposes tag-scoped token, request, dollar and concurrency limits and names the gap it fills:

No native cap on concurrent in-flight requests per tag identity

To be clear about the tool: LiteLLM is MIT-licensed and free, covers 140+ providers, and enforces hard budgets per key, team, organisation and model. A dollar budget and a concurrency cap are different instruments, and this is what the second one is for: the first bounds what has been counted, the second bounds how much can be in flight while the counting is behind.

The counting is always somewhat behind. Spend counters are fed by asynchronous pipelines — an event queue, a background emitter, a batched write — so "spent so far" trails "spent" by however long that path takes. A limit checked against a counter that lags by a second has a second's worth of traffic underneath it, whatever its configured value says.

Four things that bound the overshoot

None of these requires knowing what the next call costs.

1. A cap on output tokens for every call. With max_tokens unset, the worst case per call is the model's maximum output, and the worst case is the only number a ceiling can honestly be sized against.
2. A cap on how many calls can be in flight. The overshoot is per in-flight call, so bounding concurrency is what turns an unbounded quantity into arithmetic.
3. Headroom inside the envelope. If the worst case is N and the number you must not cross is L, the check has to refuse at L − N. A budget set at the number you actually meant will be crossed by construction.
4. A reconciliation. The pre-call figure is an estimate; the usage counts in the response are the measurement. Tracking the difference per run — see cost per run — is what tells you whether your worst case is a bound or a guess.

A Show HN the same week took the other available route and launched a prepaid, OpenAI-compatible API with fixed quotas and hard limits (thread). Prepayment does bound the worst case, at the account level: you cannot spend money that is not in the account. What it bounds is the account, not the run, and inside the balance the same unpriced-next-call problem applies to every agent drawing on it.

None of this argues for checking later

The overshoot is an argument for stating a ceiling honestly, not for moving the check. Before the call, the exposure is what is in flight; after the response, the exposure is the whole call plus however long it takes a human to read an alert, which is the subject of provider spend limits are alerts, not brakes. A limit that counts something other than money has a third gap on top of both, which run quotas and concurrency caps do not bound spend works through.

How this works in Capsera

Capsera's budget check runs in-process, before the provider call: a blocking budget raises BudgetExceededError and no tokens are spent, and throttle and margin-downgrade actions are applied at the same point. Enforcement is on by default rather than something to switch on. See pre-call enforcement for where that sits in the request lifecycle.

The limits of it, in the terms above. The check reads spend that has already been recorded, and events reach the backend through a non-blocking queue and a background emitter thread, so calls in flight during that window are not yet in the number the next check sees — the lag is small, it is not zero, and sizing an envelope means leaving room for it. A call matching no budget proceeds. Network or backend errors fail open, because a spend tool must not take down the application it is metering. Budgets are scoped per agent, per team or globally; a per-run cost ceiling is a category concept we write about, not a switch that exists today.

If what you need is per-call trace inspection, prompt content debugging or evals, Langfuse is open source, free, and built for that job. Capsera stores hashes and token counts, never prompt or response content, which is a deliberate exclusion and rules that job out for us.