A configured cost control is not an enforced one

A limit that shows up correctly on a settings page is not evidence that anything is being enforced. Between the value you typed and the request that should be refused sit a parser, a matcher, a scope and a failure mode, and each of them can drop the rule without saying so. The check that means something is behavioural: send the call that should be refused and watch what happens to it. Reading the setting back off the page you set it on tests the page.

Seven fixes shipped in one week across two open-source projects, each repairing a different way a control can be present and inert. They are worth reading as a list of the shapes this failure takes.

A setting can be parsed, displayed, then ignored

LiteLLM's fix to how it sends oversized requests to Bedrock's ApplyGuardrail API lists, among the defects it closes, this one:

A failure reported inside a 200 body was logged as success

The transport succeeded, so every layer above it recorded a pass. The check itself had not run. That pull request also records that the setting meant to bound the request size was "parsed, shown in the UI, then ignored" — the same fix, from the other side, and the reason a guardrail scan is spend you have to count separately. LiteLLM is MIT-licensed and free, and enumerating your own inert settings in a public changelog is not something most projects do.

A rule that matches on a name is disabled by a rename

Two LiteLLM changes in the same week are about rules that fail to bind. One notes that auto-router template presets "only light up when the admin's public model names literally match the preset's hardcoded names", so a proxy with renamed deployments shows presets greyed out and reporting models missing while the whole model family is in fact deployed. The other states that the Routing Groups UI "claims the group name is callable; it never was", and that a group over single-deployment model names silently does nothing.

Both are the same defect in different clothes: the control is keyed on a string someone else is free to change. Nothing errors when the string stops matching, because not matching is a legitimate state for a matcher to be in. A budget scoped to agent_name="planner" behaves identically the day someone renames the function to plan_and_route.

Scope is rarely what the switch implies

A third LiteLLM change adds a required-AND tag prefix and a fail-open flag to tag routing, and the problem statement is about blast radius rather than syntax: enable_tag_filtering is router-wide only, so opting one model group into tag routing exposes every other model group on the same proxy to the same tag evaluation. A per-group decision was in practice a per-proxy one.

This is worth pairing with a property of the same system that is documented and easy to miss: LiteLLM's tag budgets are not hierarchical, and tags are supplied per request through metadata.tags or the x-litellm-tags header. A flat namespace supplied by the caller is a perfectly reasonable design, and it means a tag budget bounds exactly the requests that remembered to carry the tag. Whether that is the set you meant is a question about your call sites, not about the budget.

A counter that is structurally always zero

CrewAI's Bedrock usage extraction reads the cache read counter under both of the names the Converse API uses, but "never looks at the matching write counters", so cache-creation tokens stayed at zero regardless of what the workload did. Two more CrewAI fixes the same week land the equivalent on the Anthropic side, where cache reads and cache writes were left out of total_tokens altogether; one of them also adds a safety net for when raw Anthropic cache keys bypass the native provider.

A metric pinned at zero is the hardest version of this to notice, because zero cache writes reads as good news. Anthropic prices cache reads at about a tenth of base input and cache writes at about 1.25×, so a stable prefix pays for itself on first reuse — which means the write counter is the one that tells you whether the cache is being populated at all. A caching strategy assessed on a field that cannot move is assessed on nothing.

What to observe instead

1. Trip the control on purpose. Set a blocking limit low enough to hit, run the workload, and confirm the call was refused rather than recorded. This is the same fault-injection method that separates fail-open from fail-closed behaviour, pointed at configuration instead of at an outage.
2. Check the counter can move in both directions. Any field showing zero should be provoked into showing something else once. A number that has never been non-zero has not been tested.
3. Ask what the rule matches on, and who can change it. Model names, group names, tags and function names are all strings owned by someone who has no reason to know a control depends on them.
4. Make silent drops countable. The constructive pattern here is Langfuse's, which on rejecting malformed telemetry reports the rejected attributes through a dedicated metric rather than discarding them quietly. Langfuse is free and open source, and a rejection you can graph is a rejection you can act on.

Where Capsera sits, including what it does not do

Capsera checks budgets before the provider call, in your process, with enforcement on by default — so the observable effect of a blocking budget is a BudgetExceededError raised where the call would have been made, and no tokens spent. Budgets are re-checked at ingest as well, which gives you a second place to see that a limit engaged. That is what pre-call enforcement buys over a report.

The parts of this post that apply to us, stated rather than omitted. A call with no matching budget proceeds; scope your budgets and then verify the scope binds, because an agent whose decorator name drifted is an agent with no envelope. Capsera fails open at every layer, so an unreachable control plane degrades to unenforced rather than to an outage — availability preserved, enforcement not. The routing rules are the same: SDK-side, pre-call, first match wins, fail open. And attribution follows the decorator, so a function that lost its @capsera.agent line reports under whatever scope encloses it, with nothing anywhere reading as an error. Trip it once and you know. Read it back and you know what the page says.