Prompt prefix stability

Provider caches match an exact prefix from the first byte: one changed character invalidates everything after it. On Anthropic's pricing, a cache read costs about a tenth of base input while a cache write costs about 1.25× — so a stable prefix pays for its one write the first time it is reused, and an unstable one pays full price forever, silently.

system = f"You are a support agent. Today is {date.today()}." puts the volatile date ahead of everything else, so no two days share a prefix. Moving the date to the end of the prompt — or out of the system prompt entirely — leaves the instruction block stable and cacheable.

By the ratio of cache-read tokens to total input tokens on real responses, and statically by counting distinct system-prompt hashes per call site: a site that should have one hash but has hundreds is rebuilding its prefix on every call.

What is prompt prefix stability?

Prompt prefix stability is the property that the opening bytes of a prompt are identical across calls, which is what allows a provider's cache to match and discount repeated input. Provider caches match an exact prefix from the first character, so one changed byte invalidates everything after it. A prompt assembled with an f-string at the top therefore has no reusable prefix at all.

• Cost defect — A cost defect is a pattern in code that makes an LLM workload cost more than the same behaviour would cost if written differently.
• Context regrowth — Context regrowth is the compounding cost of re-sending an entire conversation history with every turn, so total input tokens grow roughly with the square of the conversation length.
• Tool result bloat — Tool result bloat is the cost incurred when large tool outputs — full API payloads, whole files, verbose logs — are inserted into an agent's context and then re-sent on every subsequent turn.