AutoGen, Haystack and DSPy

All three call provider clients, so all three are recorded with no integration, and all three are on the frame-skip list so caller_file resolves to your code.

What differs is where attribution can attach.

AutoGen

Conversations run inside initiate_chat, which does not return until the exchange finishes. Attribute the conversation:

import autogen
import capsera

capsera.init(api_key=os.environ["CAPSERA_API_KEY"])

config = {"config_list": [{"model": "gpt-4o-mini", "api_key": os.environ["OPENAI_API_KEY"]}]}
reviewer = autogen.AssistantAgent(name="reviewer", llm_config=config)
user = autogen.UserProxyAgent(name="user", human_input_mode="NEVER", code_execution_config=False)


@capsera.agent("spec-review", team="engineering", task_type="review")
def review(spec: str) -> str:
    user.initiate_chat(reviewer, message=f"Review this spec:\n\n{spec}")
    return reviewer.last_message()["content"]

Every turn either agent takes is recorded and attributed to spec-review.

Per-assistant attribution is not available from outside, because both agents' calls happen inside the same loop with your scope wrapping all of it and no code of yours running between turns. Turn count is the variable that matters in AutoGen, and call count under one agent makes it visible.

Haystack

Pipelines are recorded through their generator components:

from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
import capsera

capsera.init(api_key=os.environ["CAPSERA_API_KEY"])

pipeline = Pipeline()
pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o-mini"))


@capsera.agent("kb-pipeline", team="knowledge")
def run(question: str) -> str:
    result = pipeline.run({"llm": {"prompt": question}})
    return result["llm"]["replies"][0]

Haystack pipelines often combine embedders and generators, and both are recorded. To separate them, split the pipeline run or use tag() around each stage, as described in LlamaIndex.

Embedders use the patched embeddings surface, so they bypass routing and budget enforcement. For a document-ingestion pipeline that is the bulk of the spend, which means a blocking budget will not stop it.

DSPy

DSPy modules are callables, so attribution is straightforward:

import dspy
import capsera

capsera.init(api_key=os.environ["CAPSERA_API_KEY"])
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

classify = dspy.ChainOfThought("question -> answer")


@capsera.agent("evaluator", team="research", task_type="evaluation")
def evaluate(question: str) -> str:
    return classify(question=question).answer

Give optimisation its own agent. A compiler run such as BootstrapFewShot or MIPRO makes many model calls to search for a prompt, which is a different cost from serving traffic:

@capsera.agent("dspy-compiler", team="research", task_type="optimization")
def compile_program(trainset):
    return dspy.BootstrapFewShot(metric=exact_match).compile(classify, trainset=trainset)

Without this, an optimisation run appears as a spike in the evaluation agent's cost, and the two figures you want to compare, tuning cost against serving cost, are combined.

General pattern

Where a framework owns the loop, attribute the entry point and use call count rather than per-participant cost to identify problems. Where your code is on the stack when the call happens, such as in a tool, component, or module you invoke, attribution attaches where you put it.

For finer granularity than a framework's loop allows, move the LLM call into a function of your own and decorate that.