Routing

Routing rewrites the model on a call before it reaches the provider, so eligible work runs on a cheaper model without editing call sites.

It is off by default, because it changes which model answers.

capsera.init(
    api_key=os.environ["CAPSERA_API_KEY"],
    enable_routing=True,
)

Rule types and precedence

Rules are evaluated in this order and the first match wins:

A budget under pressure overrides a task-type preference, and both override a blanket model mapping. Because the first match wins, a broad model_map rule cannot shadow a narrower task_type rule regardless of priority.

Defining rules

Rules normally live in the dashboard so they can change without a deploy. The SDK refreshes them every 60 seconds by default.

For version-controlled rules, pass them locally. A local rule is a flat dict keyed by type, not the rule_type / config_json envelope the REST API uses:

capsera.init(
    api_key=...,
    enable_routing=True,
    routing_rules=[
        {
            "type": "task_type",
            "name": "summaries-are-cheap",
            "priority": 0,
            "match": "summarization",
            "target_model": "gpt-4o-mini",
        },
        {
            "type": "model_map",
            "name": "no-opus-in-dev",
            "priority": 1,
            "from_model": "claude-opus-4-6",
            "target_model": "claude-sonnet-4-6",
        },
    ],
)

The fields each type reads:

priority orders rules within a type and defaults to the position in the list. An unrecognised type is skipped with a warning rather than raising, so a typo means that rule silently never fires — check the log the first time a local rule does nothing.

Local rules have no enabled flag. Backend rules do, and a disabled one is not sent to the SDK; to turn a local rule off, remove it.

Local and backend rules merge, and a local rule wins over a backend rule with the same name.

Guardrails

Cross-provider routing is allowed, and unguarded. A rule can move a call from Anthropic to OpenAI. Nothing stops it, and nothing yet checks that the target model supports what the call needs — a tool-using or vision call can be routed to a model that cannot do either. Providers also differ in tokenizer, message format, tool-calling shape, and refusal behaviour, so a cross-provider swap can break code that parses responses. Write cross-provider rules narrowly, and verify the target on real traffic before widening them.

Critical task types are exempt from cost-driven downgrades.

capsera.init(
    api_key=...,
    enable_routing=True,
    critical_task_types=["legal-review", "medical-summary"],
)

Calls tagged with these task types are skipped by cost_cap downgrades and by budget-margin downgrades.

Routing fails open. If rules cannot be fetched, the call proceeds on its original model.

Recorded fields

A routed call records both models:

This makes the saving computable from real events rather than projected.

Limitations

Embeddings are not routed. They bypass routing, budget enforcement, and prompt analysis.

Only the model changes. Routing does not adjust max_tokens, temperature, or any other parameter.

Quality is not evaluated. Routing will send work to a model that handles it worse. Use critical_task_types for work where that trade is unacceptable, and watch error rates per model, since recorded errors make a bad route visible.

Suggested rollout

Route one narrow, low-risk task type, observe it for a week, then widen. The original_model field lets you quantify the saving before extending it, and routing can be disabled with a single flag.