What is tool result bloat?

Tool result bloat is the cost incurred when large tool outputs — full API payloads, whole files, verbose logs — are inserted into an agent's context and then re-sent on every subsequent turn. The result is paid for once when it arrives and again on every turn that follows, so its true cost is its size multiplied by the remaining length of the session.

How do I find out which tool is inflating my agent's costs?

Measure tool-result tokens per call and compare them with completion tokens. An agent whose input is dominated by tool output — rather than by instructions or user content — is carrying payloads it probably never needed in full. Capsera extracts tool-result token counts as structural metadata on every call, without storing the content itself.

Should tool results be truncated before going into the context?

Almost always, at the tool boundary rather than in the prompt. Return the fields the task needs instead of the raw response object, cap free-text results at a size budget, and park anything large out-of-band — a file path or an ID the agent can fetch from again — rather than inlining it into history that every later turn re-pays for.

Cost defect

Tool result bloat

A tool result is not paid for once. It enters the context on the turn it arrives, and because LLM APIs re-send the whole history every call, it is paid for again on every turn after that. A 20KB JSON payload dumped into an agent’s context early in a long session costs many times its own size — position compounds size, and the tool that returned “everything, just in case” becomes the most expensive line in the transcript.

The mechanism

Agent frameworks insert each tool result into the message history so the model can reason over it. That is correct behaviour — the defect is in what gets inserted. APIs return response objects designed for programs: envelopes, pagination cursors, scoring metadata, every field of every record. A task that needs three titles receives, and then permanently carries, all of it.

Broken — the raw response object becomes permanent context

@tool
def search_products(query: str) -> str:
    resp = catalog_api.search(query)
    # The whole response object goes into the context: headers,
    # pagination, scoring metadata, twenty full records — and it will be
    # re-sent on every turn after this one.
    return json.dumps(resp.json())

From that turn on, the payload rides along with every re-send of the history. If the session runs another fifteen turns, the object is billed fifteen more times. Agents that loop over tools — search, read, search again — stack several of these payloads into the same growing context.

How to tell if it's happening to you

The signal is the ratio of tool-result tokens to everything else in the input. Capsera’s prompt analysis extracts tool-result token counts per call as structural metadata — no prompt content is stored — so the agents whose context is mostly tool output stand out directly, and its suggestion rules flag the tool-result-heavy ones. Without instrumentation, the manual version is to log the length of each tool’s return value and multiply by the number of turns that follow it: that product, not the length, is the cost.

What it costs

Arithmetic, not a benchmark: a tool result of ~5,000 tokens returned on turn three of an eighteen-turn session is re-sent fifteen times — roughly 75,000 input tokens for one payload the model needed once. Trim it to the ~200 tokens the task actually consumed and the same session sends ~3,000. The saving scales with both payload size and how early in the session the tool fires, which is why the worst offenders are retrieval tools called at the start of long sessions.

How to fix it

Fix it at the tool boundary, where the payload is still a value in your code, rather than trying to prompt the model into ignoring what it was sent:

Fixed — return what the task needs, capped

RESULT_BUDGET = 1_200  # characters; tune per tool, enforce at the boundary

@tool
def search_products(query: str) -> str:
    resp = catalog_api.search(query)
    hits = resp.json()["results"][:5]
    # Return what the task consumes, not what the API happened to send.
    lines = [f'{h["id"]}: {h["name"]} — {h["price"]}' for h in hits]
    out = "\n".join(lines)
    return out[:RESULT_BUDGET]

Select fields instead of serialising the response object. Cap the size with an explicit budget so a pathological result cannot flood the context. And for genuinely large artifacts — files, tables, long documents — park them out of band: return a path or an ID the agent can fetch a slice of on demand, so the full content never becomes history at all.

Questions this page answers

What is tool result bloat?: Tool result bloat is the cost incurred when large tool outputs — full API payloads, whole files, verbose logs — are inserted into an agent's context and then re-sent on every subsequent turn. The result is paid for once when it arrives and again on every turn that follows, so its true cost is its size multiplied by the remaining length of the session.
How do I find out which tool is inflating my agent's costs?: Measure tool-result tokens per call and compare them with completion tokens. An agent whose input is dominated by tool output — rather than by instructions or user content — is carrying payloads it probably never needed in full. Capsera extracts tool-result token counts as structural metadata on every call, without storing the content itself.
Should tool results be truncated before going into the context?: Almost always, at the tool boundary rather than in the prompt. Return the fields the task needs instead of the raw response object, cap free-text results at a size budget, and park anything large out-of-band — a file path or an ID the agent can fetch from again — rather than inlining it into history that every later turn re-pays for.