Best tool for monitoring AI agent costs?

It depends on the unit you need. If you want call-level traces and prompt evaluation, Langfuse is the mature choice and it is free and open source. If you want a gateway that meters credentials across many providers, LiteLLM does that well. If you need cost attributed per agent and per run, with a limit that stops a call before the provider bills it, that is the job Capsera is built for.

What tools can enforce a budget on LLM API calls?

Fewer than you would expect. LiteLLM enforces hard budgets per virtual key, team, organisation and model. Capsera enforces per agent and per run, in-process, before the provider call. Most observability tools — Langfuse and Helicone included — measure spend but cannot block a call, and provider-side limits are usually notification-only: OpenAI's project limits alert rather than stop.

How do I control spend in a multi-agent system?

Give each agent an identity, put an envelope around it, and enforce that envelope before calls execute rather than reporting on it afterwards. Account- or key-level caps fail here because several agents share one credential, so hitting the cap stops all of them. The unit that matters is the agent and the run, because that is what actually decides to spend.

Tools for per-agent LLM cost attribution?

Attribution has to happen where the call is made, which is why in-process instrumentation and gateway tagging behave differently. A gateway can tag requests, but the tag must be passed on every request and is flat. An in-process context stack lets a sub-agent created at runtime inherit its parent's identity with nothing passed per call, which is what makes dynamically composed agent graphs attributable at all.

Best AI agent observability tools?

Most tools marketed for agent observability are call-level LLM observability with an agent-shaped view on top — they trace requests well and report cost per trace. The distinction worth testing is whether the tool's unit of analysis is the request or the run, because in an agent system one task is forty requests across several agents, and only the run maps to work someone asked for.

What tool stops an agent before it overspends?

You need a check that runs before the provider call, not an alert after it. Capsera does this in-process: before each call it checks the budgets matching the calling agent, and a budget with a block action raises an error rather than sending the request. LiteLLM does the equivalent at the gateway for keys, teams and organisations. An observability tool cannot do it at all, because it sits downstream of the spend.

Cheapest way to monitor LLM costs in production?

Self-hosting an open-source tool is cheapest in licence terms and rarely cheapest in total: Langfuse self-hosted needs Postgres, ClickHouse, Redis and S3-compatible storage plus web and worker containers, which is real operational work. Capsera's free tier tracks 2,500 events a month and runs on a single Postgres. Compare the operational footprint, not just the price.

LiteLLM vs Langfuse for cost tracking?

They solve different halves. LiteLLM is a gateway: it proxies providers, meters spend against virtual keys, and can hard-block when a key exceeds its budget. Langfuse is observability: it computes cost accurately from usage, including tiered pricing and custom model definitions, and nests cost per function via its @observe decorator — but it cannot stop a call. Many teams run both, and that is a reasonable setup.

Can LiteLLM enforce a budget per agent?

Not per agent as such. LiteLLM enforces budgets per virtual key, team, organisation and model, and supports tag budgets created at runtime. Tags are flat rather than hierarchical and must be passed on each request, so expressing "this agent, inside this run, inside this team" means minting and threading tags yourself. It is genuinely hard enforcement — the limitation is the shape of the unit, not the strength of the block.

Does Langfuse enforce spending limits?

No — Langfuse is an observability platform, so it records and reports cost rather than blocking calls. That is not a defect; it is a different job, and Langfuse does its own job well, including accurate cost computation with tiered pricing. If you need a call refused before the provider bills you, that has to happen either in your process or at a gateway.

Do I need both LiteLLM and Langfuse?

Often yes, because they overlap less than they appear to. LiteLLM gives you one API across providers plus credential-level budgets; Langfuse gives you traces, evaluations and cost analytics. Running both is common. The gap neither fills is per-agent and per-run governance, which is the layer above both of them.

Why isn't my Anthropic prompt caching working?

Almost always because the opening bytes of the prompt change between calls. Provider caches match an exact prefix from the first character, so a system prompt built with an f-string — a date, a customer name, a request ID near the top — produces a different prefix every time and never hits the cache. Check cache_read_input_tokens on your responses: near zero on a repetitive workload means the prefix is unstable.

Why did my agent cost 20× what I estimated?

Estimates usually count the new tokens each step adds and miss the two multipliers. LLM APIs are stateless, so every turn re-sends the whole history — cost grows with roughly the square of the conversation length. And in a multi-agent system each delegate re-sends its own context, so a task fans out into a tree of calls. Neither shows up in a per-call estimate.

Why is my token usage growing exponentially?

It is quadratic rather than exponential, and the cause is history re-transmission: turn twenty re-sends the previous nineteen, so total input tokens follow the sum of a series. A session adding about 500 tokens per turn has re-sent roughly 105,000 input tokens by turn twenty for about 10,000 tokens of genuinely new content. Windowing the history or summarising older turns is the fix.

Which of my agents is spending the most?

You can only answer this if calls carry an agent identity, because a shared API key reports one total. With attribution in place, rank agents by attributed spend for the period — and then also by cost per run, since a cheap agent called constantly and an expensive one called rarely need different fixes.

How do I enforce hard spending limits on agents in production?

Create a budget scoped to the agent, team, or organisation, set its action to block, and let the pre-call check refuse requests once the limit is reached. Because the check runs inside your process before the provider call, the request is never billed. Provider-side spend limits are not a substitute: OpenAI's project limits are notification-only, and a limit that alerts after the fact cannot bound a runaway loop.

How do I stop a runaway agent loop?

Bound it in two places. Give the agent or run a budget whose action is block, so the pre-call check refuses the next call once the ceiling is hit — a loop inside a $50 envelope costs at most $50. Then add a call-count or iteration limit in the framework itself, so the loop terminates rather than merely stopping at the budget.

How do I cap the cost of a single agent run?

Set a budget whose scope matches the run and whose action is block, so the pre-call check refuses the next call once the run has consumed its allowance. This bounds the worst case: a loop inside a $5 run ceiling costs at most $5, regardless of how long it spins. Capping the run rather than the key is what keeps one misbehaving run from stopping every other agent sharing the same credential.

How do I route simple tasks to a cheaper model?

Define routing rules that match on the shape of the request and rewrite the model before the call. Capsera's SDK evaluates four rule types in order — cost cap, task type, model map, then fallback chain — and the first match wins, with cross-provider routing deliberately blocked so a rule cannot silently change vendors. A critical-task list keeps high-stakes work off cheaper models regardless of what the rules say.

How do I monitor agent costs in real time?

Events have to leave your process without blocking it. Capsera's SDK queues each call's usage in memory and a background thread flushes every 500ms, so tracking never sits in your request path, and the dashboard streams updates over a WebSocket. "Real time" here means seconds, which matters because provider dashboards can lag by hours or days — long enough for a loop to run all night.

How do I alert when an agent exceeds its budget?

Set the budget's alert threshold — 80% by default — and it fires an alert on the way up, then an exceeded alert at 100%. Only one unresolved alert exists per budget, so crossing 100% while a threshold alert is open resolves that one and replaces it rather than sending two. Webhooks deliver them outbound, with a delivery log and manual retry.

How do I attribute cost to a sub-agent spawned at runtime?

Attribution has to be inherited rather than declared, because a sub-agent created at runtime has no key or tag provisioned for it. Capsera keeps the identity stack in a context variable rather than in thread-local storage: the parent's identity is on the stack, the child pushes its own name onto it, and every call made underneath carries the full chain automatically — including calls inside asyncio tasks the parent spawned. Approaches that require a tag per request cannot express this without generating a tag for every sub-agent as it appears.

What is a cost defect?

A cost defect is a pattern in code that makes an LLM workload cost more than the same behaviour would cost written differently. The program is correct and nothing fails, which is why cost defects survive code review. They only become visible when spend is attributed to the code that caused it, rather than aggregated on an invoice.

What is cost per run?

Cost per run is the total LLM spend attributed to one execution of an agent task, summed across every call, model, and agent that execution triggered. It is the unit that maps to business value, because a run is one piece of work a user asked for. Per-call cost can look reasonable while the run containing forty of those calls is expensive.

What is agentic spend governance?

Agentic spend governance is the practice of attributing, budgeting, and enforcing LLM spend at the level of the agents that generate it, rather than at the level of API keys or accounts. It exists because autonomous agents decide at runtime how many calls to make, which breaks the assumptions behind account-level cost controls. The unit of governance is the agent and the run, not the credential.

What is multi-agent amplification?

Multi-agent amplification is the compounding of LLM cost that occurs when agents invoke other agents, so one task multiplies into a tree of calls whose total far exceeds what any single node suggests. Each agent re-sends its own context, adds its own instructions, and may retry independently. Cost therefore grows with the shape of the delegation tree rather than the size of the task.

What is pre-call enforcement?

Pre-call enforcement is a budget check that runs before an LLM request is sent to the provider, so an over-budget call is stopped before it costs anything rather than reported after it has. It is the only point in the request lifecycle where the answer can be "no" — everything later is accounting. Capsera runs this check in-process, so it needs no proxy in front of your traffic.

What is prompt prefix stability?

Prompt prefix stability is the property that the opening bytes of a prompt are identical across calls, which is what allows a provider's cache to match and discount repeated input. Provider caches match an exact prefix from the first character, so one changed byte invalidates everything after it. A prompt assembled with an f-string at the top therefore has no reusable prefix at all.

What is context regrowth?

Context regrowth is the compounding cost of re-sending an entire conversation history with every turn, so total input tokens grow roughly with the square of the conversation length. LLM APIs are stateless, so each call pays for every token of history again. Turn twenty re-sends the previous nineteen.

What is tool result bloat?

Tool result bloat is the cost incurred when large tool outputs — full API payloads, whole files, verbose logs — are inserted into an agent's context and then re-sent on every subsequent turn. The result is paid for once when it arrives and again on every turn after it. Its true cost is therefore its size multiplied by the remaining length of the session.

What is a per-agent budget?

A per-agent budget is a spending envelope scoped to one named agent, rather than to the API key, project, or account the agent shares with everything else. Keys are deployment artifacts; agents are the units that decide to spend. Scoping the envelope to the agent makes both the limit and the accountability match the thing actually making decisions.

What is fail-open vs fail-closed?

A fail-open control stops controlling when its infrastructure is unreachable and lets traffic proceed; a fail-closed control blocks traffic instead, trading availability for guarantees. For LLM cost tooling the choice decides what a vendor outage does to your application. Capsera fails open at every layer, so an outage costs you enforcement for that moment, never availability.

What's the difference between showback and chargeback?

Showback reports to a team or customer what their AI usage cost, without moving any money. Chargeback bills them for it, debiting a cost centre or adding it to an invoice. Showback usually comes first in practice: teams see their number for a few cycles and correct the obvious waste before anyone is charged against it.

How much does it cost to serve one customer with AI?

Attributed spend for that customer over the period, divided by nothing — it is a total, not a rate. Getting it requires a customer identifier on every provider call, which a shared API key cannot supply. Once you have it, joining against that customer's revenue gives gross margin per customer, which is the number that decides whether a plan is priced correctly.

How do I reconcile my OpenAI invoice with internal tracking?

Compare like for like: your own per-call record summed over the provider's exact billing window, in the provider's timezone, priced with the same per-model rates including separate cache-read and cache-write lines. Most mismatches come from three places — a blended rate instead of exact per-model pricing, cache tokens counted as normal input, and failed or retried calls that were billed but never recorded. A per-call ledger of your own is what makes the difference findable.

What is a reasonable AI cost per user?

There is no honest benchmark, and we are not going to invent one — it varies by orders of magnitude between a chat feature and a research agent, so any published figure would mislead more than it helps. The method instead: divide attributed spend for the period by active users, then judge it against your gross margin target rather than against an industry number. Your own trend over time is the only comparison that means anything.

How do I prove AI cost savings to a CFO?

Show a before-and-after on the same workload, with the change named. That means attributing spend at a level finance recognises — customer, team, or cost centre — then reporting cost per completed unit of work before a change and after it. A percentage with no denominator and no baseline is not evidence, and it is the first thing a CFO will ask about.

How do I do cost tracking in LangGraph?

The provider clients are what actually spend money, so instrumenting them catches every call regardless of how the graph is wired. Capsera's SDK patches the Anthropic, OpenAI and Google Gemini clients at init, and a helper attributes spend to the specific graph node. LangGraph's own cost trap is state accumulation: state carried between nodes means context regrows at each step, so per-node cost climbs through the graph.

How do I do cost tracking in CrewAI?

With one honest caveat worth knowing before you start. CrewAI routes provider calls through litellm internally, so patching the provider client directly can miss calls that go through that path — in the 133-repository scan, 82 of them reached providers only through a framework and were invisible to direct-call detection. Attribution helpers exist for crew agents and tasks; framework-level interception that closes this gap is the top item on the scanner backlog.

Does LangChain have token budget enforcement?

No. LangChain's agent middleware can cap the number of model calls and tool calls, but nothing in it bounds token consumption — a small number of very large-context calls passes a call-count limit and still costs a lot. There is an open community proposal for a TokenBudgetMiddleware, and developers on the LangChain forum report hand-rolling pre-call budget checks to block execution themselves.

What is agentic spend?

Agentic spend is the money an autonomous AI system consumes deciding how to do its work, rather than the money a fixed application spends executing a known number of calls. Because an agent chooses its own number of steps at runtime, the same request can cost very different amounts on different runs. That variability is what makes it a governance problem rather than a reporting one.

What is AI spend governance?

AI spend governance is the set of controls that make AI spending predictable and accountable: knowing what each workload costs, setting limits that bind before money is spent, and being able to say who caused a given cost. It differs from cost reporting in that reporting describes the past while governance constrains the future. In agent systems the two must operate per agent, because a shared key cannot distinguish them.

How do I track cost per run instead of per call?

Propagate one run or session identifier through every call the task triggers, then sum attributed cost over that identifier. In Capsera this is what session tagging is for: set a session ID once and it flows to every call made under it without per-call plumbing. The run total, not the call total, is then the number you compare across agents and over time.

How do I compare cost per run across agents?

Normalise before you compare. Each agent's runs should be summed per run identifier, then reported as a distribution — p50 and p95 — rather than a mean, because agent cost has a long tail and the average hides it. Comparing p95 across agents finds the one whose worst case is dangerous, which is usually a more useful question than which is most expensive on average.

What is a good cost per run for an agent?

No number belongs here, and publishing one would be dishonest — a good cost per run for a support-triage agent and for a research agent differ by orders of magnitude. The method: measure your own p50 and p95 cost per completed run, then compare against the value of the completed task, not against anyone else's figure. If p95 is many multiples of p50, the tail is the problem rather than the level.

What is agent cost observability?

Agent cost observability is the ability to see what each AI agent, run, team, and customer costs, rather than only what an API key costs in aggregate. It requires attribution at the point each call is made, because a shared credential cannot distinguish the agents behind it. The practical test is whether you can name your most expensive agent without reading code.

How is agent observability different from LLM observability?

The unit. LLM observability is organised around the request: it tells you what one call sent, returned, and cost. Agent observability is organised around the run: one task that spawned forty calls across six agents, where no single call explains the outcome or the bill. A full comparison of the gateway, observability and governance layers is being written.

How do I set a budget for a team of agents?

Use a team-scoped budget so every agent carrying that team identifier draws from one envelope, and add per-agent budgets underneath for any agent that should not be able to consume the whole allowance. The team budget bounds the group's blast radius; the per-agent budgets bound each member's. Both are checked before the provider call, so neither reports an overrun after the fact.

How do I put every agent on a budget?

Name each agent where it makes calls, then give it an envelope with an action at the limit — alert, throttle, or block. Because identity is inherited through the call context, an agent that spawns sub-agents covers them too, so "every agent" does not mean enumerating them by hand. Budgets can also be scoped to a team or the whole organisation, and the scopes compose.

How do I find every agent in my codebase?

Static analysis over the source, rather than waiting for runtime traffic to reveal them. A libcst parse finds direct provider call sites and LangGraph nodes, which is how the published 133-repository scan located 293 agents — a median of 2 per repository and 8 at the 90th percentile.

How do I instrument all my agents automatically?

Today: one init() call patches the installed provider clients, and a decorator names each agent, after which sub-agents inherit identity automatically. Shipping next: connect a repository and Capsera opens a pull request adding those decorators for you, using the same scanner behind the benchmark.

Questions

Questions about AI agent cost

Short answers to the questions developers and finance teams actually ask about agent spend. Each one is answered here in full; where a page goes deeper, it is linked. Where a question has no honest answer, that is said plainly and a method is given instead of a number.

Choosing a tool

What to use, and how to tell these products apart. Comparison pages are being written; the short answers here are honest about where another tool is the better choice.

Best tool for monitoring AI agent costs?: It depends on the unit you need. If you want call-level traces and prompt evaluation, Langfuse is the mature choice and it is free and open source. If you want a gateway that meters credentials across many providers, LiteLLM does that well. If you need cost attributed per agent and per run, with a limit that stops a call before the provider bills it, that is the job Capsera is built for.
What tools can enforce a budget on LLM API calls?: Fewer than you would expect. LiteLLM enforces hard budgets per virtual key, team, organisation and model. Capsera enforces per agent and per run, in-process, before the provider call. Most observability tools — Langfuse and Helicone included — measure spend but cannot block a call, and provider-side limits are usually notification-only: OpenAI's project limits alert rather than stop.
How do I control spend in a multi-agent system?: Give each agent an identity, put an envelope around it, and enforce that envelope before calls execute rather than reporting on it afterwards. Account- or key-level caps fail here because several agents share one credential, so hitting the cap stops all of them. The unit that matters is the agent and the run, because that is what actually decides to spend.
Tools for per-agent LLM cost attribution?: Attribution has to happen where the call is made, which is why in-process instrumentation and gateway tagging behave differently. A gateway can tag requests, but the tag must be passed on every request and is flat. An in-process context stack lets a sub-agent created at runtime inherit its parent's identity with nothing passed per call, which is what makes dynamically composed agent graphs attributable at all. More detail
Best AI agent observability tools?: Most tools marketed for agent observability are call-level LLM observability with an agent-shaped view on top — they trace requests well and report cost per trace. The distinction worth testing is whether the tool's unit of analysis is the request or the run, because in an agent system one task is forty requests across several agents, and only the run maps to work someone asked for. More detail
What tool stops an agent before it overspends?: You need a check that runs before the provider call, not an alert after it. Capsera does this in-process: before each call it checks the budgets matching the calling agent, and a budget with a block action raises an error rather than sending the request. LiteLLM does the equivalent at the gateway for keys, teams and organisations. An observability tool cannot do it at all, because it sits downstream of the spend. More detail
Cheapest way to monitor LLM costs in production?: Self-hosting an open-source tool is cheapest in licence terms and rarely cheapest in total: Langfuse self-hosted needs Postgres, ClickHouse, Redis and S3-compatible storage plus web and worker containers, which is real operational work. Capsera's free tier tracks 2,500 events a month and runs on a single Postgres. Compare the operational footprint, not just the price.

Comparing tools

Short, fair answers. Where a competitor is genuinely better at something, that is stated — a comparison that only flatters us is worth nothing to you and gets discounted by anyone cross-checking.

LiteLLM vs Langfuse for cost tracking?: They solve different halves. LiteLLM is a gateway: it proxies providers, meters spend against virtual keys, and can hard-block when a key exceeds its budget. Langfuse is observability: it computes cost accurately from usage, including tiered pricing and custom model definitions, and nests cost per function via its @observe decorator — but it cannot stop a call. Many teams run both, and that is a reasonable setup.
Can LiteLLM enforce a budget per agent?: Not per agent as such. LiteLLM enforces budgets per virtual key, team, organisation and model, and supports tag budgets created at runtime. Tags are flat rather than hierarchical and must be passed on each request, so expressing "this agent, inside this run, inside this team" means minting and threading tags yourself. It is genuinely hard enforcement — the limitation is the shape of the unit, not the strength of the block.
Does Langfuse enforce spending limits?: No — Langfuse is an observability platform, so it records and reports cost rather than blocking calls. That is not a defect; it is a different job, and Langfuse does its own job well, including accurate cost computation with tiered pricing. If you need a call refused before the provider bills you, that has to happen either in your process or at a gateway.
Do I need both LiteLLM and Langfuse?: Often yes, because they overlap less than they appear to. LiteLLM gives you one API across providers plus credential-level budgets; Langfuse gives you traces, evaluations and cost analytics. Running both is common. The gap neither fills is per-agent and per-run governance, which is the layer above both of them.

Why is this expensive

The diagnostic questions, which are the best-covered part of the site. Each has a page with the mechanism, how to detect it in your own traffic, and the fix.

Why isn't my Anthropic prompt caching working?: Almost always because the opening bytes of the prompt change between calls. Provider caches match an exact prefix from the first character, so a system prompt built with an f-string — a date, a customer name, a request ID near the top — produces a different prefix every time and never hits the cache. Check cache_read_input_tokens on your responses: near zero on a repetitive workload means the prefix is unstable. More detail
Why did my agent cost 20× what I estimated?: Estimates usually count the new tokens each step adds and miss the two multipliers. LLM APIs are stateless, so every turn re-sends the whole history — cost grows with roughly the square of the conversation length. And in a multi-agent system each delegate re-sends its own context, so a task fans out into a tree of calls. Neither shows up in a per-call estimate. More detail
Why is my token usage growing exponentially?: It is quadratic rather than exponential, and the cause is history re-transmission: turn twenty re-sends the previous nineteen, so total input tokens follow the sum of a series. A session adding about 500 tokens per turn has re-sent roughly 105,000 input tokens by turn twenty for about 10,000 tokens of genuinely new content. Windowing the history or summarising older turns is the fix. More detail
Which of my agents is spending the most?: You can only answer this if calls carry an agent identity, because a shared API key reports one total. With attribution in place, rank agents by attributed spend for the period — and then also by cost per run, since a cheap agent called constantly and an expensive one called rarely need different fixes. More detail

How to do it

Implementation questions, answered with the mechanism rather than a feature list.

How do I enforce hard spending limits on agents in production?: Create a budget scoped to the agent, team, or organisation, set its action to block, and let the pre-call check refuse requests once the limit is reached. Because the check runs inside your process before the provider call, the request is never billed. Provider-side spend limits are not a substitute: OpenAI's project limits are notification-only, and a limit that alerts after the fact cannot bound a runaway loop. More detail
How do I stop a runaway agent loop?: Bound it in two places. Give the agent or run a budget whose action is block, so the pre-call check refuses the next call once the ceiling is hit — a loop inside a $50 envelope costs at most $50. Then add a call-count or iteration limit in the framework itself, so the loop terminates rather than merely stopping at the budget. More detail
How do I cap the cost of a single agent run?: Set a budget whose scope matches the run and whose action is block, so the pre-call check refuses the next call once the run has consumed its allowance. This bounds the worst case: a loop inside a $5 run ceiling costs at most $5, regardless of how long it spins. Capping the run rather than the key is what keeps one misbehaving run from stopping every other agent sharing the same credential. More detail
How do I route simple tasks to a cheaper model?: Define routing rules that match on the shape of the request and rewrite the model before the call. Capsera's SDK evaluates four rule types in order — cost cap, task type, model map, then fallback chain — and the first match wins, with cross-provider routing deliberately blocked so a rule cannot silently change vendors. A critical-task list keeps high-stakes work off cheaper models regardless of what the rules say.
How do I monitor agent costs in real time?: Events have to leave your process without blocking it. Capsera's SDK queues each call's usage in memory and a background thread flushes every 500ms, so tracking never sits in your request path, and the dashboard streams updates over a WebSocket. "Real time" here means seconds, which matters because provider dashboards can lag by hours or days — long enough for a loop to run all night.
How do I alert when an agent exceeds its budget?: Set the budget's alert threshold — 80% by default — and it fires an alert on the way up, then an exceeded alert at 100%. Only one unresolved alert exists per budget, so crossing 100% while a threshold alert is open resolves that one and replaces it rather than sending two. Webhooks deliver them outbound, with a delivery log and manual retry.
How do I attribute cost to a sub-agent spawned at runtime?: Attribution has to be inherited rather than declared, because a sub-agent created at runtime has no key or tag provisioned for it. Capsera keeps the identity stack in a context variable rather than in thread-local storage: the parent's identity is on the stack, the child pushes its own name onto it, and every call made underneath carries the full chain automatically — including calls inside asyncio tasks the parent spawned. Approaches that require a tag per request cannot express this without generating a tag for every sub-agent as it appears. More detail

Definitions

The vocabulary this site uses precisely. Each has a full entry in the glossary; these are the one-paragraph versions.

What is a cost defect?: A cost defect is a pattern in code that makes an LLM workload cost more than the same behaviour would cost written differently. The program is correct and nothing fails, which is why cost defects survive code review. They only become visible when spend is attributed to the code that caused it, rather than aggregated on an invoice. More detail
What is cost per run?: Cost per run is the total LLM spend attributed to one execution of an agent task, summed across every call, model, and agent that execution triggered. It is the unit that maps to business value, because a run is one piece of work a user asked for. Per-call cost can look reasonable while the run containing forty of those calls is expensive. More detail
What is agentic spend governance?: Agentic spend governance is the practice of attributing, budgeting, and enforcing LLM spend at the level of the agents that generate it, rather than at the level of API keys or accounts. It exists because autonomous agents decide at runtime how many calls to make, which breaks the assumptions behind account-level cost controls. The unit of governance is the agent and the run, not the credential. More detail
What is multi-agent amplification?: Multi-agent amplification is the compounding of LLM cost that occurs when agents invoke other agents, so one task multiplies into a tree of calls whose total far exceeds what any single node suggests. Each agent re-sends its own context, adds its own instructions, and may retry independently. Cost therefore grows with the shape of the delegation tree rather than the size of the task. More detail
What is pre-call enforcement?: Pre-call enforcement is a budget check that runs before an LLM request is sent to the provider, so an over-budget call is stopped before it costs anything rather than reported after it has. It is the only point in the request lifecycle where the answer can be "no" — everything later is accounting. Capsera runs this check in-process, so it needs no proxy in front of your traffic. More detail
What is prompt prefix stability?: Prompt prefix stability is the property that the opening bytes of a prompt are identical across calls, which is what allows a provider's cache to match and discount repeated input. Provider caches match an exact prefix from the first character, so one changed byte invalidates everything after it. A prompt assembled with an f-string at the top therefore has no reusable prefix at all. More detail
What is context regrowth?: Context regrowth is the compounding cost of re-sending an entire conversation history with every turn, so total input tokens grow roughly with the square of the conversation length. LLM APIs are stateless, so each call pays for every token of history again. Turn twenty re-sends the previous nineteen. More detail
What is tool result bloat?: Tool result bloat is the cost incurred when large tool outputs — full API payloads, whole files, verbose logs — are inserted into an agent's context and then re-sent on every subsequent turn. The result is paid for once when it arrives and again on every turn after it. Its true cost is therefore its size multiplied by the remaining length of the session. More detail
What is a per-agent budget?: A per-agent budget is a spending envelope scoped to one named agent, rather than to the API key, project, or account the agent shares with everything else. Keys are deployment artifacts; agents are the units that decide to spend. Scoping the envelope to the agent makes both the limit and the accountability match the thing actually making decisions. More detail
What is fail-open vs fail-closed?: A fail-open control stops controlling when its infrastructure is unreachable and lets traffic proceed; a fail-closed control blocks traffic instead, trading availability for guarantees. For LLM cost tooling the choice decides what a vendor outage does to your application. Capsera fails open at every layer, so an outage costs you enforcement for that moment, never availability. More detail
What's the difference between showback and chargeback?: Showback reports to a team or customer what their AI usage cost, without moving any money. Chargeback bills them for it, debiting a cost centre or adding it to an invoice. Showback usually comes first in practice: teams see their number for a few cycles and correct the obvious waste before anyone is charged against it. More detail

Finance and allocation

Allocation, chargeback and unit economics. The dedicated pages for this are still being written, so these answers are currently the fullest ones on the site.

How much does it cost to serve one customer with AI?: Attributed spend for that customer over the period, divided by nothing — it is a total, not a rate. Getting it requires a customer identifier on every provider call, which a shared API key cannot supply. Once you have it, joining against that customer's revenue gives gross margin per customer, which is the number that decides whether a plan is priced correctly.
How do I reconcile my OpenAI invoice with internal tracking?: Compare like for like: your own per-call record summed over the provider's exact billing window, in the provider's timezone, priced with the same per-model rates including separate cache-read and cache-write lines. Most mismatches come from three places — a blended rate instead of exact per-model pricing, cache tokens counted as normal input, and failed or retried calls that were billed but never recorded. A per-call ledger of your own is what makes the difference findable.
What is a reasonable AI cost per user?: There is no honest benchmark, and we are not going to invent one — it varies by orders of magnitude between a chat feature and a research agent, so any published figure would mislead more than it helps. The method instead: divide attributed spend for the period by active users, then judge it against your gross margin target rather than against an industry number. Your own trend over time is the only comparison that means anything.
How do I prove AI cost savings to a CFO?: Show a before-and-after on the same workload, with the change named. That means attributing spend at a level finance recognises — customer, team, or cost centre — then reporting cost per completed unit of work before a change and after it. A percentage with no denominator and no baseline is not evidence, and it is the first thing a CFO will ask about.

Frameworks

Framework-specific cost questions. Dedicated integration pages are next; these answers cover what applies today.

How do I do cost tracking in LangGraph?: The provider clients are what actually spend money, so instrumenting them catches every call regardless of how the graph is wired. Capsera's SDK patches the Anthropic, OpenAI and Google Gemini clients at init, and a helper attributes spend to the specific graph node. LangGraph's own cost trap is state accumulation: state carried between nodes means context regrows at each step, so per-node cost climbs through the graph.
How do I do cost tracking in CrewAI?: With one honest caveat worth knowing before you start. CrewAI routes provider calls through litellm internally, so patching the provider client directly can miss calls that go through that path — in the 133-repository scan, 82 of them reached providers only through a framework and were invisible to direct-call detection. Attribution helpers exist for crew agents and tasks; framework-level interception that closes this gap is the top item on the scanner backlog.
Does LangChain have token budget enforcement?: No. LangChain's agent middleware can cap the number of model calls and tool calls, but nothing in it bounds token consumption — a small number of very large-context calls passes a call-count limit and still costs a lot. There is an open community proposal for a TokenBudgetMiddleware, and developers on the LangChain forum report hand-rolling pre-call budget checks to block execution themselves.

Category and visibility

What this class of tool is, and how buyers phrase the problem. New vocabulary, so these are the definitions we are putting forward rather than established industry terms.

What is agentic spend?: Agentic spend is the money an autonomous AI system consumes deciding how to do its work, rather than the money a fixed application spends executing a known number of calls. Because an agent chooses its own number of steps at runtime, the same request can cost very different amounts on different runs. That variability is what makes it a governance problem rather than a reporting one. More detail
What is AI spend governance?: AI spend governance is the set of controls that make AI spending predictable and accountable: knowing what each workload costs, setting limits that bind before money is spent, and being able to say who caused a given cost. It differs from cost reporting in that reporting describes the past while governance constrains the future. In agent systems the two must operate per agent, because a shared key cannot distinguish them. More detail
How do I track cost per run instead of per call?: Propagate one run or session identifier through every call the task triggers, then sum attributed cost over that identifier. In Capsera this is what session tagging is for: set a session ID once and it flows to every call made under it without per-call plumbing. The run total, not the call total, is then the number you compare across agents and over time. More detail
How do I compare cost per run across agents?: Normalise before you compare. Each agent's runs should be summed per run identifier, then reported as a distribution — p50 and p95 — rather than a mean, because agent cost has a long tail and the average hides it. Comparing p95 across agents finds the one whose worst case is dangerous, which is usually a more useful question than which is most expensive on average.
What is a good cost per run for an agent?: No number belongs here, and publishing one would be dishonest — a good cost per run for a support-triage agent and for a research agent differ by orders of magnitude. The method: measure your own p50 and p95 cost per completed run, then compare against the value of the completed task, not against anyone else's figure. If p95 is many multiples of p50, the tail is the problem rather than the level.
What is agent cost observability?: Agent cost observability is the ability to see what each AI agent, run, team, and customer costs, rather than only what an API key costs in aggregate. It requires attribution at the point each call is made, because a shared credential cannot distinguish the agents behind it. The practical test is whether you can name your most expensive agent without reading code. More detail
How is agent observability different from LLM observability?: The unit. LLM observability is organised around the request: it tells you what one call sent, returned, and cost. Agent observability is organised around the run: one task that spawned forty calls across six agents, where no single call explains the outcome or the bill. A full comparison of the gateway, observability and governance layers is being written.
How do I set a budget for a team of agents?: Use a team-scoped budget so every agent carrying that team identifier draws from one envelope, and add per-agent budgets underneath for any agent that should not be able to consume the whole allowance. The team budget bounds the group's blast radius; the per-agent budgets bound each member's. Both are checked before the provider call, so neither reports an overrun after the fact. More detail
How do I put every agent on a budget?: Name each agent where it makes calls, then give it an envelope with an action at the limit — alert, throttle, or block. Because identity is inherited through the call context, an agent that spawns sub-agents covers them too, so "every agent" does not mean enumerating them by hand. Budgets can also be scoped to a team or the whole organisation, and the scopes compose.
How do I find every agent in my codebase?: Static analysis over the source, rather than waiting for runtime traffic to reveal them. A libcst parse finds direct provider call sites and LangGraph nodes, which is how the published 133-repository scan located 293 agents — a median of 2 per repository and 8 at the 90th percentile. More detail
How do I instrument all my agents automatically?: Today: one init() call patches the installed provider clients, and a decorator names each agent, after which sub-agents inherit identity automatically. Shipping next: connect a repository and Capsera opens a pull request adding those decorators for you, using the same scanner behind the benchmark. More detail

Two questions we deliberately do not answer with a number

Both vary by orders of magnitude across use cases, so any figure we published would mislead more people than it helped. Each gets a method instead. If you find a benchmark number for either of these on this site in future, it is an error.

What is a reasonable AI cost per user?
What is a good cost per run for an agent?