A daily cost rollup cannot answer per-run questions

A daily spend total answers one question — what did we spend yesterday — and cannot be made to answer another. Cost per run, cost per agent, and whether the cache held across one conversation are not sums of a day's rows; they are properties of a unit that does not line up with a calendar day. Which questions you can ask is fixed when you choose the grain of the row you store, not when you open the dashboard.

A session does not close on a day boundary

The clearest statement of this came out of LiteLLM last week, in a pair of pull requests building a backend for their auto-router benchmarks tab. The first describes the constraint without hedging:

Session metrics cannot come from the daily rollups: session_id is not on them and a session does not close on a day boundary

Two separate failures in one sentence. The first is the join: the rollup key does not carry the identifier, so no amount of arithmetic over those rows recovers per-session spend. The second survives even if you add the identifier — a conversation that starts at 23:40 and runs an hour lands in two buckets, and the two halves cannot be added back together without knowing they belong to the same thing.

Agent runs have the same shape as sessions and are worse about it. A long multi-turn run spans hours, retries and several agents. Our July 2026 scan found a median of 2 agents per repository and 8 at the 90th percentile (method and limitations) — so the unit a finance team wants to price is routinely a thing that crosses both agent boundaries and clock boundaries.

Order is the second thing a sum destroys

A daily total by model tells you what share of yesterday's tokens went to the expensive tier. It cannot tell you whether that share came from one escalation held for the rest of a run or from a router alternating tiers turn after turn — two shapes with very different cache behaviour and different fixes. Addition is commutative; the thing you want to know is not.

The rebuilt rollup PR puts a price on recovering it afterwards. Classifying a single turn, it notes, "needs the session's prior state" — same model as last time, first visit to a tier, or a return to one already used. A summed row has no prior state. It has one number per bucket, and the sequence that produced it was discarded at write time by the thing doing the summing.

That is the read-side view of an argument this journal has already made from the other end: anything a call knows about itself is cheapest to record at the call and progressively harder to reconstruct later. Tracking tokens is not the same as tracking cost works through what goes wrong inside the derivation. The point here is upstream of that — the row's grain decides which derivations remain possible at all.

The raw log is the fallback, and the raw log is the part you cannot keep

The usual answer to "the rollup doesn't have it" is to scan the raw events. That is exactly what the earlier of the two PRs is trying to stop doing — every figure the tab needs:

can only be had today by scanning LiteLLM_SpendLogs, which is the widest table in the schema and unbounded at customer scale

Unbounded tables get trimmed. Whatever you have to retain raw rows to compute, you will eventually stop being able to compute, on a schedule set by storage cost rather than by anyone's reporting needs. The rollup is the thing that survives — so any identity missing from its key is not merely inconvenient to query, it is permanently gone for every period already closed.

To be fair to both projects in this space: LiteLLM is MIT-licensed and free, covers 140+ providers, and enforces hard budgets per key, team, organisation and model. Langfuse calculates costs from ingested and inferred usage, supports tiered pricing and custom model definitions, and nests cost per function with its @observe decorator. Neither of these is a gap in a tool; it is a schema decision every cost system makes once, early, and lives with.

Finance allocates in dollars, per something

The grain question shows up again at month end, in a different vocabulary. A comment on Hacker News about who controls AI budgets puts the constraint plainly:

Project managers and real executives manage a budget in dollars, not man days or tokens.

Dollars per what, though, is the part the schema decided. If your rollup key is day, model and API key, the only allocation you can defend is per API key — a deployment artefact that maps to a cost centre by convention and gets remapped whenever someone rotates a credential. Chargeback and showback both need the business identity on the row, not reconstructed from a naming scheme afterwards.

The engineering side has the mirrored problem. In the thread on Databricks cutting its AI coding spend, a commenter argued where the savings actually come from:

I think the real savings come from careful context control for programmatic agents, careful tool awareness and usage to reduce thrashing, distilling workflows into deterministic processes

Every one of those is a per-run behaviour. A daily total by model can tell you the number moved. It cannot attribute the movement to any of them, because the thing that changed — how one run behaved — was aggregated away before anyone looked.

Four decisions to make before you aggregate

1. Name the unit before the key. Cost per call, cost per run and cost per completed task are three different products, and the key you pick is a commitment to one of them.
2. Put identity on the event, not in the join. Run identifier, agent, team, customer, cost centre — written at request time, when the process still knows them.
3. Keep sequential facts as fields. Which model served this turn, whether input was a cache read or a cache write, which rule chose the tier. These are free to record and expensive to infer.
4. Assume the raw rows expire. For each question you expect to ask next quarter, check it survives on the rollup alone. The ones that do not are the fields you are missing.

How this works in Capsera

Capsera is in-process rather than a gateway, which is what makes the identity available at write time: capsera.init() patches the provider clients, and a @capsera.agent decorator on a function you already have attributes everything called inside it. Because the attribution stack is a ContextVar, nested agents inherit it and asyncio.gather does not interleave the identities. Every event therefore leaves the process already carrying its agent, run, session, customer and cost-centre identity, rather than being matched to one later.

That is also the constraint behind per-agent budgets. A budget can only be scoped to an agent if the spend is, and enforcement happens before the provider call, not against yesterday's rollup. What Capsera does not store is prompt or response content — hashes and token counts only — so the row is narrow enough to keep at the grain the questions need. If the job you actually have is call-level tracing or evals, that is Langfuse's; the layering is set out in gateway vs observability vs governance.