The answers you discard are still on the bill

A router that buys quality by running one request through several models bills you for every arm, not for the answer you keep. Three model groups in parallel plus a synthesizer that merges them is four billed calls behind one client-visible response, and not one of those four looks unusual on its own. The number of arms is set in the router's configuration rather than in your code, so the average cost per call barely moves while the cost of a completed task multiplies.

The fan-out is a router feature now

A pull request opened on 2026-09-01 adds a best_of_n router with parallel fan-out and a synthesizer to LiteLLM. It states the gap it fills:

No way to trade cost for quality above a single frontier model

and the mechanism:

New best_of_n/ deployment fans a request to N configured model groups in parallel

A configured synthesizer merges the candidates into the final answer

A second pull request the same day, adding fusion_router virtual models, describes a wider fan and names the accounting shape in passing:

when deliberation is invoked, run 1–8 configured panel models in parallel, ask an analyst for structured consensus/contradictions, and return that advisory result to the outer model for the only client-visible response or tool call

Both are proposals in an open repository, not shipped behaviour. What matters is the shape they converge on, because it is the shape people are asking for. A comment on Hacker News on 2026-09-04, under a thread about fast inference, puts the demand side plainly:

Isn't context size one of the central motivations of the whole agent / orchestration business - fanning out increasingly detailed work to a tree of subagents.

Every arm is a normal-looking call

The discarded candidates are not retries, errors, or waste in the sense a diagnostic tool looks for. Each one is a well-formed request to a model you chose, returning tokens you asked for. Deliberate redundancy is the product.

That is what makes the multiplier hard to see from a call-grained view. If three arms run the same frontier model, the mean cost per call is roughly what it was before and the total per answer is around four times it — so a dashboard reporting cost per call reports almost nothing, having divided a much larger total by a much larger denominator. This is not multi-agent amplification, where each node of a delegation tree does different work; here the arms do the same work and all but one is thrown away on purpose. It is also not the cost of an offline experiment: these arms sit in the serving path, on every request that takes them.

The arithmetic matters because it competes directly with the savings people count. A day-0 release note on 2026-09-02 for Meta's Muse Spark 1.3 on LiteLLM states that Meta "reports roughly 20% fewer tool calls and 25% fewer tokens than 1.2 internally". Take that at face value: one arm of the new model costs three quarters of one arm of the old one. Two arms of it cost 0.75 × 2 = 1.5 — half again more than the single call it replaced, with the model efficiency gain fully spent and then some. The token-efficiency work and the fan-out decision land on the same line item, and only one of them is usually reviewed.

The counters hang off the client, not the agent

A CrewAI fix from 2026-09-04, counting each LLM instance once when summing usage metrics, quotes its own documented behaviour:

The counters are cumulative for the lifetime of this instance: they grow across every call made through it, including calls issued by different agents sharing the instance.

A fan-out shares client objects almost by construction — several arms against one model group is one configured client, used repeatedly. A cumulative counter read after the fact answers "how much went through this object", which is not the same question as "what did this arm cost". The fix counts each instance once, and that repairs the crew total while giving up the per-arm split in the same move: with no per-call event carrying an arm identity, there is nothing to split by. Whether a discarded candidate counts as an attempt or as its own operation is the same denominator question a retry raises, and it has to be answered before the ratio means anything.

The multiplier lives in someone else's configuration

The fusion proposal above keeps "the normal model API", and a third pull request on 2026-09-03, limiting how many auto-routers a proxy may run, reports the status quo it changes:

Any proxy can run unlimited heuristic_v2 auto-routers from config.yaml or the API

Your application asks for one model name. How many models answer is a line in a configuration file owned by whoever operates the proxy. Moving one arm to three is a cost change of roughly the size of a model swap, and it appears in no diff of yours.

What to record

1. Billed calls per client-visible answer, per run. If that ratio is not one, cost per call is measuring a different thing than you think.
2. Arm identity at call time — which arm, which model group, kept or discarded. Working out afterwards which of four simultaneous calls was the one you kept is not a join that exists.
3. The discarded arms as their own line. That total is the price of the quality premium. Reported separately it is a decision; folded into a model's cost it looks like the model got more expensive.
4. Who owns the number of arms, and a review when it changes.
5. A ceiling on the concurrent worst case, because a pre-call check prices one call at a time — a budget check cannot price the call it allows.

How this works in Capsera

Where the fan-out happens decides what an in-process SDK can see, and the answer has two halves.

If the arms are issued from your process — asyncio.gather over several clients, or a framework's own best-of-N step — capsera.init() has already patched the provider clients, so every arm is captured without a call site changing, and @capsera.agent(name="synthesizer") supplies the identity through contextvars. Because the agent stack is a ContextVar rather than thread-local state, one decorator scope holds across asyncio.gather instead of interleaving, and nested agents inherit, so an arm reports as itself under its parent run. Budgets are scoped per agent, per team or globally and checked before the provider call, so a blocking budget raises BudgetExceededError and the arm spends nothing.

If the fan-out happens inside a gateway, the arms are calls the gateway makes, in the gateway's process, and Capsera sees the single request you sent. That is a real limit of running in-process instead of in the request path. The arm-level rows exist in the gateway's own spend log — the best_of_n proposal above states that "Every arm and the synthesizer log their own real spend…" — so the work is reconciling one outbound request against N rows over there. LiteLLM is MIT-licensed, free, covers 140+ providers, and enforces hard budgets per key, team, organisation and model, which is where to bound a fan-out it owns. And if the question is which candidate won and why — the content of the arms rather than their cost — Langfuse is open source, free, and built for that; Capsera stores hashes and token counts, never prompt or response content.