Compacting a context costs twice

Compacting a long transcript is the standard answer to a context that has grown too big, and it is billed twice before it saves anything. The compaction itself is a model call that reads the transcript at the largest it has ever been, and the summary it writes back replaces a prefix the provider's cache was already matching, so the next turn pays to write a new one. Both charges land up front. Whether the run comes out cheaper depends on how many turns are left to spread them over, which is a number nobody knows at the moment the compaction fires.

This is the other half of context regrowth. That page is about the history growing. This one is about what the remedy costs.

The trigger is quality, not spend

A Show HN on 2026-10-28, a garbage collector for LLM context windows, states the complaint the way practitioners actually experience it:

On cloud models, as the context grows, the model starts getting noticeably dumber. IMHO the root problem is context bloat and the fact that compaction destroys the quality

Read that as a cost statement and it has three separate quantities in it: the model degrades as input grows, compaction is the thing you reach for, and the compaction has a cost of its own that the author pays in answer quality. The same post reports the local-inference version as latency — "after a few hours prefill takes 2 to 3 minutes per turn" — which is the same re-read, billed in seconds on hardware you own rather than in tokens on someone else's.

Nobody in that thread is running a cost review. The decision to compact is made because the output got worse or the turn got slow, and the bill moves as a side effect.

The compaction call is the largest call in the run

A summarisation pass is a request whose input is everything the run has accumulated up to the trigger point — by construction, the biggest prompt the run has assembled. If a loop compacts three times, three of its requests sit near that high-water mark and none of them produces work the user asked for.

They are also calls no line of your application code issues directly — the harness or the framework issues them on your behalf, which is what makes them sit in the same category as the calls you never wrote. A per-call average over the run hides them by dividing them into the turns around them.

The summary invalidates the prefix it replaces

The second charge is the cache. Provider prompt caching matches a prefix from the first byte; see prompt prefix stability for what that depends on. A compaction rewrites the middle of the history, so every byte after the rewrite no longer matches what was cached, and the turn immediately after a compaction is billed as a cache write rather than a cache read.

That is the mechanical cost. The shape it produces is a sawtooth: input tokens per call climb, drop at the compaction, then climb again from a floor that is higher than the last one, because the summary is carried forward alongside the new turns. Compaction trades a quadratic for a sequence of smaller quadratics. It does not reset the run.

Deleting the inputs is the same decision, made earlier

The cheaper version of this move is to not put the material in the context at all, and two threads the same week describe teams doing exactly that for non-cost reasons.

A commenter on 2026-10-27, in a thread on specification-first agentic development, describes removing the documents:

we've started to actively remove all spec documents so agents don't get stuck reviewing out of date context for 20 minutes, burn through token budgets, and give disproportionate focus to legacy patterns and business directions we've pivoted from

Three costs in one sentence: the tokens, the wall-clock, and the misdirected work a stale document buys. The last is the expensive one — a run steered toward a pattern the team abandoned is billed in full and then thrown away.

The other version trims what tools hand back. A submission on 2026-10-28, on wrapping CLIs to make them more token-efficient, describes a wrapper that "prepares the most relevant information for agents—so the LLM doesn't have to filter it out itself", and the author is straightforwardly undecided about whether it pays:

I haven't yet formed an opinion on whether this is overengineering or a genuine way to save time and money

That is the honest position, and it is the position almost everyone is in, because the comparison requires measuring the same task twice. Trimming tool output is adjacent to the fixed cost of the tool catalog itself, and the two are measured the same way.

The arithmetic, over assumed numbers

None of this is a measurement — it is arithmetic over inputs you would supply yourself.

Take a 60-turn run that compacts whenever the transcript passes some threshold, and say that fires three times. Three requests are issued at or near the transcript's maximum size, and three turns immediately afterwards are billed as uncached input instead of cache reads. Against that, the turns between compactions each carry a smaller history than they otherwise would.

The saving is the difference between two totals, and it is sensitive to where the compactions land: one that fires on the second-to-last turn is pure overhead, and one that fires a quarter of the way through has forty turns to earn its cost back. Nothing in a per-call view separates those two cases — both are a handful of calls among sixty.

It also hides the defect it is fixing

The published detection method for context regrowth is to watch input tokens per call over the life of a session and look for a line sloping upward. Compaction breaks that signal. The curve resets at every compaction, so a session that is regrowing just as fast as before reads as a series of short, well-behaved climbs.

The quantities that survive are cumulative input tokens per run and the number of compactions the run performed. If a run compacted four times and its cumulative input is still higher than a comparable run that compacted none, the compaction was not a saving — it was a different way of paying.

What to record

1. Compactions per run, as a count. It is the clearest single indicator of a loop that has outgrown its context budget, and it is not derivable from a spend total afterwards.
2. Cumulative input tokens per run, not the per-call slope. The slope is the quantity compaction resets.
3. Cache reads, cache writes and uncached input as separate dimensions. A compaction shows up as a write where a read was expected, and collapsing them into one input figure deletes the evidence.
4. The turn index each compaction fired on. Position decides whether it paid for itself, and it is not recoverable from the total.
5. Cost per run, on runs that compacted and runs that did not. Per-call figures are the wrong unit for a change that lowers the price of many calls while adding a few expensive ones.

How this works in Capsera, and what it does not do

Capsera captures the calls, including the ones you did not write. capsera.init() patches the provider clients — Anthropic messages, OpenAI chat completions, OpenAI embeddings and Google Gemini, sync and async — so a summarisation request issued by a framework inside your process is recorded like any other, with its input, output and cache token counts kept separate. @capsera.agent(name="planner") adds who, not whether, and because the agent stack is a contextvars ContextVar rather than thread-local state, nested agents inherit the identity, so a compaction that happens underneath an agent reports under that agent and under the run that caused it.

The limitation is worth stating plainly: Capsera stores hashes and token counts, never prompt or completion content, so it cannot tell you that a particular call was a compaction rather than a turn of work. What it gives you is the call, its size and its cache dimensions under the right agent — you still have to decide which calls were context management. And budgets are checked before the provider call, scoped per agent, per team or globally, which bounds a loop that keeps compacting and regrowing without requiring anyone to predict how often it will.

Two places to go elsewhere. If the question is what was in the transcript at the moment it was compacted — the content, not its size — Langfuse is open source, free and built to store and inspect exactly that, and Capsera is not, by design. If you need spend capped per key, team or organisation across many providers, LiteLLM is MIT-licensed, free, and enforces hard budgets at those scopes.