OpenAI-compatible providers

Any provider that implements the OpenAI API is recorded with no extra setup, because it uses the openai client the SDK patches. Coverage is full: sync, async, streaming, and errors.

To tell these providers apart, the SDK reads the client's base_url and derives the provider from its host. Without that, all of them would be reported as openai and provider-level cost comparison would be impossible.

Recognised hosts

import openai
import capsera

capsera.init(api_key=os.environ["CAPSERA_API_KEY"])

groq = openai.OpenAI(
    base_url="https://api.groq.com/openai/v1",
    api_key=os.environ["GROQ_API_KEY"],
)

groq.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Classify this ticket."}],
)

That call is recorded with provider="groq" and priced from Groq's catalog entry.

Local models

Ollama on its default port is recognised, so local inference is attributed rather than mislabelled:

local = openai.OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

Local models usually have no marginal cost, so the value here is token-volume visibility. A model the catalog does not know is recorded with a fallback estimate and a warning. For local inference that estimate is not meaningful, so consider record() with cost_usd=0.0.

vLLM behaves the same way. On a non-standard host it is not recognised by name and falls back as described below.

Unrecognised hosts

A private gateway or self-hosted endpoint on an unlisted host is still recorded. Provider labelling falls back to the catalogued provider for the model name, and then to openai.

A call to https://llm.internal.example.com/v1 using gpt-4o-mini is therefore recorded as openai, which is correct about the model and says nothing about the gateway. To distinguish an internal gateway in reports, report the host and it will be added to the map.

Model pricing

The catalog covers the main model lines for Groq, DeepSeek, xAI, Together, Fireworks, and Perplexity. Prefix matching means a dated or suffixed variant inherits its base model's price.

These vendors add and retire models faster than a fixed table tracks. When you see the fallback-estimate warning, the call was recorded with an approximate cost. Report the model and it will be added.

Prompt caching

Cache-token accounting is Anthropic-specific in this SDK. OpenAI-compatible vendors with their own caching semantics report standard input and output tokens only, so a cached prefix appears as ordinary input. Your bill from that vendor may be lower than Capsera's figure.