Context regrowth: the quadratic re-read

Turn one sends the system prompt and one message. Turn two sends both of those plus the model’s reply and the new message. Turn twenty re-sends the previous nineteen. Nothing about this is a provider quirk or a bug — it is how a stateless completion API has to work — but it means the cost of a conversation is not the sum of its turns; it is the sum of its re-reads.

Agents make it worse than chat in two ways. Tool loops add turns no human reads — each tool result becomes history that every later step re-pays for. And multi-agent pipelines run several of these growing contexts at once, one per agent.

A worked example — arithmetic, not a benchmark. Suppose each turn adds about 500 tokens (a message, a reply, or a tool result). By turn n, each new call re-sends roughly n × 500 tokens, so the running total of input tokens is about 500 × n(n+1)/2:

Doubling the conversation length roughly quadruples the input bill. That non-linearity is why the defect hides so well: the first week of a feature’s life, sessions are short and the numbers look fine.

Plot input tokens per call across the life of a session. Flat workload, flat line: healthy. A steady upward slope is history growing under you. The two structural signals are conversation turn count and context-window utilisation per call — both of which Capsera’s prompt analysis extracts as metadata, without storing any prompt content. An agent whose median call uses 60% of the context window is one long session away from truncation errors as well as the bill.

In order of cheapness:

Reset between tasks.The most common source of accidental regrowth is reusing one session for independent jobs. If task two doesn’t need task one’s transcript, start clean.

Window and summarise. Keep the last N turns verbatim and compress everything older into a digest the agent maintains as it goes:

MAX_TURNS = 12  # tune per task; most agent steps don't need deep history

def build_messages(history: list[dict], new_user_msg: dict) -> list[dict]:
    # Keep the newest turns verbatim; replace everything older with the
    # running summary the agent maintains as it completes sub-tasks.
    recent = history[-MAX_TURNS:]
    older = history[:-MAX_TURNS]
    messages = []
    if older:
        messages.append({
            "role": "user",
            "content": f"Summary of the conversation so far: {summarise(older)}",
        })
    return messages + recent + [new_user_msg]

Cache what you keep. A conversation history is an append-only prefix — exactly the shape provider caches reward. With a stable prefix, the re-read still happens but is billed at cache-read pricing (about 0.1× base input on Anthropic) instead of full price. Caching softens the quadratic; it does not remove it.

Why did my agent cost far more than the token estimate?

Most estimates count the new tokens each turn adds and forget that LLM APIs are stateless: every call re-sends the entire history so far. A conversation that adds a constant amount per turn costs the sum of an arithmetic series — roughly the square of the turn count — so a 20-turn session where each turn adds about 500 tokens has re-sent around 105,000 input tokens for about 10,000 tokens of new content.

How do I detect context regrowth in production?

Watch input tokens per call over the life of a session. On a flat workload the curve should be flat; a line that slopes steadily upward is history being re-sent and growing. Conversation turn count and context-window utilisation per call are the two structural signals — Capsera extracts both without storing prompt content.

How do I stop conversation history from growing without losing quality?

Three standard moves, cheapest first: reset context between independent tasks instead of reusing one long session; window the history to the last N turns when older turns stop mattering; and summarise completed work into a short digest that replaces the verbatim turns. Prompt caching also softens the price of what you do keep, since a stable history prefix is served at cache-read pricing.

• Tool result bloat — the fastest-growing ingredient of the history being re-sent
• Broken prompt caching — why the re-read is billed at full price
• All cost defects