Glossary
Cost defect
A cost defect is a pattern in code that makes an LLM workload cost more than the same behaviour would cost if written differently.
Why it matters
It is not a bug — the program produces correct output — and it is not a pricing problem. Because nothing fails, cost defects are invisible in code review and absent from provider dashboards, which report totals rather than causes. They surface only when spend is attributed to the code that generated it.
Example
A system prompt assembled with an f-string changes on every call, so the provider's prefix cache never matches and every request pays full input price. In Capsera's July 2026 scan of 133 public agent repositories, 76% of the 51 making direct LLM calls contained at least one cost defect of this kind.
How it's measured
By static analysis of call sites (the approach behind the benchmark) or at runtime, by comparing what a call cost against what the same tokens would have cost with the defect fixed — for caching defects, the gap between cache-read pricing and full input pricing.
Questions this page answers
- What is a cost defect?
- A cost defect is a pattern in code that makes an LLM workload cost more than the same behaviour would cost written differently. The program is correct and nothing fails, which is why cost defects survive code review. They only become visible when spend is attributed to the code that caused it, rather than aggregated on an invoice.
- Why don't I have visibility into my agent costs?
- Because provider billing is organised by API key, and an API key is a deployment artifact rather than a unit of work. One key typically serves many agents, so the provider can report a total but cannot say which agent, run, or customer produced it. Getting visibility means attributing each call at the point it is made, inside your own process, to the agent that made it.