A looping agent looks healthy to your monitoring

Almost every signal an infrastructure monitor collects is derived from failure: error rate, restart count, latency spikes, queue depth, saturation, failed health checks. An agent stuck in a loop produces none of them. It issues well-formed requests that come back 200, waits on the network with the CPU near idle, never crashes and never retries. The service is healthy by every definition the monitoring stack holds, and the only quantity moving is the one that stack does not count.

Two Hacker News items on 2026-11-06 describe that shape from opposite ends — one as a question about a new runtime, one as operating advice.

A loop is made of successes

A commenter on the Launch HN thread for an agent orchestration runtime asks about the failure mode directly:

Curious how Rivet handles the runaway loop problem though. When you have long-lived actors passing messages to each other, a simple state bug can cause them to ping-pong the exact same payload indefinitely.

And names what the operator sees while it happens:

Because the actors stay alive and keep responding (returning 200s), your infra looks perfectly healthy

Go through the layers one at a time and the claim holds. The HTTP client sees successful responses. The provider SDK sees no rate-limit errors worth backing off from, because a loop that is waiting on each response in turn is politely paced. The process supervisor sees a process that has not exited. The autoscaler sees low CPU, because the work is network-bound. An uptime check sees an endpoint that answers. Nothing in that list is wrong, and nothing in it is watching the only thing that changed.

This is what separates a runaway agent loop from the outages those tools were built for. An outage is a shortage of something the host can see and reclaim — memory, file descriptors, CPU, disk. Tokens are bought from a third party on credit, outside the process, and consuming more of them makes every local resource graph look better, not worse. It is the same asymmetry behind the fact that a sandbox bounds what an agent touches, not what it spends: the one capability a containment layer has to allow is the one that bills.

An always-on agent has no run boundary to bound

Cost per run needs a run that ends. A long-lived actor, a daemon, a polling worker or a durable session has no terminal state by design — it is supposed to still be there tomorrow. Every ceiling that is enforced at the end of something therefore never fires, and every figure that is reported per completed unit has no unit to report against.

Hibernating an idle actor, which is the right move for compute, is inverted for tokens. An actor stays resident exactly as long as messages keep arriving, so the one state in which it is cheap to host is the state in which it spends nothing, and the one state in which it bills is the state the platform is designed to keep alive. Scale-to-zero is a compute control that cannot reach this case, because this case never goes idle.

What is left to measure is a rate rather than a total. For an agent with no run boundary, tokens per hour is the primary series and cost per run is a derived one — available only where you have minted a run identifier yourself, above whatever loop the runtime is spinning.

Waking on an event changes the trigger, not the ceiling

The other item is an answer in an Ask HN thread about building an internal knowledge agent, and it gives the standard mitigation (2026-11-06):

When the system is up and running don't run agents in a loop, wake them up when something happens. An email arrives, something changed, a timer triggered. There is no reason for 24x7 agents most of the time.

That is good advice and it removes a real cost: the polling turns that ask whether there is anything to do and are billed for the answer "no". It does not bound what the wake-ups cost. The arrival rate of the triggering events is now the multiplier, and it is set outside your process — by however many emails land, by a webhook that retries, by a batch job that writes a thousand rows and emits a thousand change events. If one agent's own output is what writes the record that wakes it, the loop is back with an event bus in the middle of it, and now it spans two systems, so neither one's logs show the cycle.

Event-driven agents also fragment the accounting. Fifty wakes in an hour are fifty short-lived contexts, each with its own first uncached call, and a daily rollup will show them as one number that looks like steady background usage.

Watching it is not stopping it

A third comment, in a thread about agent reliability on 2026-11-04, puts the limit of the observing layer plainly:

You can improve the observability how much you want, it will never prevent agents to go off the rails and destroy the database

The spend version of that sentence is the one this site keeps arriving at. Dashboards, usage pages and spend alerts are all artefacts produced after a response came back, which means the tokens they describe are already bought — provider spend limits are alerts, not brakes. Reducing the lag from a day to a minute reduces the size of the bill and does not change which side of the call the control is on. The only check that can refuse spend is one that runs before the request leaves the process: pre-call enforcement.

What to record for an agent that never finishes

1. Tokens per unit time, per agent. For anything long-lived this is the primary series. A total since start is monotonic and tells you nothing; a rate has a shape, and a step change in it is the event you want to catch.
2. Billed calls since task state last advanced. A loop's signature is model calls accumulating while the artefact they are supposed to be producing does not change. That counter is application-specific and nothing else can derive it for you.
3. Repeat rate. How many of the last N requests were byte-identical to an earlier one. Ping-ponging the same payload is the cheapest runaway to detect and needs no content, only a hash per request.
4. Wakes per hour and spend per wake for event-driven agents, kept as two numbers. One number hides which half moved.
5. A declared expected rate per always-on agent, written down when it is deployed. Anomaly detection against an undeclared baseline learns whatever the loop has been doing all week and calls it normal.

How this works in Capsera, and what it does not do

capsera.init() patches the provider clients — Anthropic messages, OpenAI chat completions, OpenAI embeddings and Google Gemini, sync and async — so calls issued from inside a long-running worker are captured without a request path change or a per-call edit. @capsera.agent(name="watcher") says which agent they belong to, and because the agent stack is a contextvars ContextVar rather than thread-local state, the identity holds per thread and per asyncio task — which is what an actor runtime or an event consumer is made of.

Capsera stores hashes and token counts, never prompt or completion content. That is a limitation for most questions and an advantage for this one: request hashes make a repeated payload countable without a transcript, which is what the repeat-rate signal above needs. The dashboard carries burn rate, exhaustion-date forecasting and anomaly detection over those series.

Budgets are checked before the provider call and are on by default. They are scoped per agent, per team or globally, with alert, throttle and block actions; a blocking budget raises BudgetExceededError and no tokens are spent. Two things that does not give you. There is no per-run cost ceiling — the scope is the agent, which for an always-on worker is the right grain anyway, since the run is the part that does not exist. And enforcement fails open: a network or backend error lets the call through, which is the correct default for a library in your request path and the wrong assumption to build a hard guarantee on.

If what you need is the trace content — the actual messages the two actors were exchanging, to find the state bug rather than to bound its cost — Langfuse is open source, free, and built to store and inspect exactly that. Capsera is not, by design.