Get 2,500 events tracked for freeSign up now

All posts
Muhammad Kumail5 min readBudgets, Attribution

The caller decides how many turns you pay for

When another company's bot opens your support chat, it picks how many model turns you buy. Why LLM governance for inbound agents is a budget question.

Nearly every cost control on an agent assumes your side started the run. A scheduler fired, a developer typed a prompt, a user you hold an account for clicked something. Put an agent behind a surface anyone can reach — a support chat widget, an inbox autoresponder, a free-tier API — and the number of runs becomes an input set by whoever is on the other end. Increasingly that is somebody else's outbound agent, and a ceiling built for work you initiate has to hold against a quantity you do not choose.

Something else is opening your chat widget

An Ask HN thread posted on 2026-11-16 describes the traffic arriving (source):

The SaaS I work for has started being spammed with startup ads. Except now they aren't just emails, they're AI agents visiting our website, opening the help desk chat widget in the corner, and spamming their message.

The widget is the interesting part. An ad in an inbox costs the recipient attention. A conversation opened against a support agent costs the recipient model calls, because the surface was built to answer — it reads the message, decides whether to answer or escalate, and probably re-reads the whole thread to do it. The sender pays for their side; you pay for yours; neither party has visibility into the other's bill, and only one of you chose to have the conversation.

Two more threads the same day show the volume this arrives at. One commenter reports the change on a submission thread (source):

The other day my show HN has resulted in maybe 20 sincere sounding AI pitches all carefully trying to approach me in just the right way.

And in a thread about scraped contact details, the suggested response is itself an agent (source):

you could probably set an LLM with the right prompt to write a set of plausible replies to bait them into sending the payload.

That last one is the shape worth looking at directly: an automated sender and an automated replier, each one's turn count determined entirely by how long the other keeps responding.

A bot does not get bored

The implicit bound on an inbound conversation has always been human patience. A person with a real question asks it, gets an answer and leaves; a person without one leaves sooner. Nothing in the system enforced that. It was a property of who was typing.

An outbound agent has the opposite property. It is running a task, the task is to get a reply, and a reply is exactly what your support agent is built to produce. Each side's output is the other side's trigger, so the exchange is a loop whose participants sit in different organisations. Neither side's iteration cap bounds it, because neither side is iterating: each one is answering what it received. From your infrastructure's point of view this looks like healthy traffic for the same reason a looping agent looks healthy — every request is valid, every response is a 200, and nothing in the availability stack has an opinion about volume that arrives on purpose.

The usual control here is a rate limit per session or per address, and it is worth being precise about what that buys. A limit expressed in requests bounds requests. The spend per request is set by the transcript length, the tool catalogue and the model, none of which the limiter sees, which is the argument in run quotas and concurrency caps do not bound spend with the quantity now chosen by someone adversarial to your interests rather than merely indifferent to them.

A per-conversation turn cap has a sharper version of the same gap. It bounds one conversation. What you are billed for is that cap multiplied by the number of conversations, and only the first factor is yours.

The spend has no customer

Cost attribution on a product surface normally resolves to an account: this run belongs to that customer, that customer's plan covers it, the margin is the difference. An inbound prospecting bot has no account. It is not a trial, it is not a lead you are courting, and it will never appear in the denominator of any unit-economics figure you report.

That makes it a case the reporting usually handles by accident. Spend with no identity attached either lands in a catch-all bucket that nobody reads, or gets spread across real customers as overhead, which quietly worsens every margin per customer by an amount set by strangers. Neither is a measurement. The honest treatment is a named line: unattributed inbound spend, reported as its own quantity, never allocated.

Where the ceiling has to sit

LLM governance for a surface like this means deciding, before the traffic arrives, what the agent is allowed to spend over a window regardless of who asks — because the identity you can key the limit to is the agent, not the caller. A caller-keyed ceiling needs the caller to be identifiable, and an unauthenticated widget is reachable precisely because it is not. The agent is the one thing in the exchange you own, which is the general point in the identity a budget is keyed to decides what it stops, arriving here because the alternative identity does not exist.

The quantities worth recording:

  1. Billed model turns per inbound conversation, as a distribution. The mean is set by humans asking real questions. The tail is the thing you are paying for, and an average hides it by construction.
  2. Conversations per originator over a window, where an originator is whatever weak key you have — address, domain, session fingerprint. It will not be reliable. It does not need to be, to show a shape.
  3. Turns taken after the conversation stopped advancing, counted separately from turns before. A support exchange has a goal state; a prospecting exchange does not, and the turn index where answers stop converging is the signal.
  4. Spend with no account identity attached, as its own line in the report rather than as a residue in someone else's.

How this works in Capsera, and what it does not do

capsera.init() patches the provider clients — Anthropic messages, OpenAI chat completions, OpenAI embeddings and Google Gemini, sync and async — so every model call in the process is captured wherever it was made. @capsera.agent(name="support") says which agent the calls inside it belong to, and nested agents inherit that identity through a contextvars ContextVar rather than thread-local state. capsera.set_session(session_id) puts one identifier on every call in a conversation, which is what turns item 1 in that list from an aggregate into a distribution. A per-agent budget at agent, team or global scope is checked before the provider call and on by default, so a blocking budget on the public-facing agent raises BudgetExceededError with no tokens spent — the difference between alerting and stopping.

The limits are worth stating plainly. A per-agent budget that blocks is a blunt instrument on a customer-facing surface: it stops the bot and it stops the real support conversation happening at the same moment, because the ceiling is scoped to the agent and the traffic is mixed. Throttle and margin-downgrade actions exist for that reason and are the more useful shape here. There is no per-run cost ceiling in the product — per-agent budgets are what ships. We store hashes and token counts and never prompt or response content, so Capsera cannot tell you which conversations read like a pitch; that classification has to happen in your own code, before the message reaches the model. And the vendor fees an inbound conversation can trigger downstream are not denominated in tokens at all.

If you need to read the actual message content to work out which inbound conversations were bots, Langfuse is open source, free, and stores exactly what we deliberately do not. If the surface is an HTTP API whose path you control, a gateway is the right place for a per-credential spend log, and LiteLLM is MIT-licensed and self-hostable.

The cost of a run you start is a design decision. The cost of a run someone else starts is a design decision too — you just have to make it before they arrive.

Give every agent an identity, a budget, and hard limits.

One line of code. Anthropic, OpenAI, and Google Gemini.

See pricing