Agent Observability with Langfuse

Agent observability with Langfuse means capturing every LLM call, tool call, and decision an autonomous agent makes as a structured trace, so when the agent does something wrong, you can open one screen and see exactly which step broke instead of re-reading raw logs. Agents are not single-shot chat completions. A single user request can trigger a planner call, three tool calls, a retrieval step, a sub-agent handoff, and a final synthesis call, and if any one of those goes sideways the end user just sees "the agent gave a weird answer." Without tracing, you are debugging blind. This article walks through setting up Langfuse for a real agent, instrumenting nested spans, tracking sessions across multi-turn conversations, scoring outputs, and wiring cost and latency dashboards you can actually act on.

Why agent observability is different from LLM logging

Logging a single prompt and response is trivial: you print the input, print the output, maybe log token counts. Agent observability with Langfuse is a different problem because an agent run is a tree, not a line. A user message kicks off a root trace. That trace fans out into spans: a "plan" generation, a "retrieve_context" tool call, a "call_calculator" tool call, maybe a nested sub-agent that itself makes three more LLM calls. Each of these needs its own timing, its own token usage, its own input/output pair, and a parent-child relationship back to the root trace so you can reconstruct the whole execution path later.

The other difference is that agents fail in ways plain chat completions don't:

• Tool call arguments get hallucinated (the model calls get_weather(city="Pariss"))
• The agent loops, calling the same tool repeatedly without making progress
• A sub-agent silently returns an error string that the parent agent treats as valid data
• Latency compounds across steps until a "fast" model still produces a 40-second response
• Cost compounds the same way, and nobody notices until the monthly bill

None of these show up in a single request/response log. You need the full trace tree, with timing and cost attached to every node, to catch them. That's the gap Langfuse is built to fill, and it's open source, so you can self-host it or use their managed cloud without changing your instrumentation code.

Setting up Langfuse for an agent project

Start with a Python agent project. Install the SDK and set your credentials as environment variables.

pip install langfuse

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="https://cloud.langfuse.com"

If you're self-hosting, point LANGFUSE_HOST at your own instance instead. The SDK is transport-agnostic; nothing else in your code changes.

Langfuse's current SDK is built on OpenTelemetry under the hood, which means traces and spans follow the OTel data model: every trace has a root span, spans can nest arbitrarily deep, and each span carries attributes (input, output, metadata, timing). You get this for free through Langfuse's decorators and context managers, so you rarely touch OTel directly unless you're integrating with an existing OTel pipeline.

The fastest way to verify the wiring works is a one-call smoke test:

from langfuse import Langfuse

client = Langfuse()

with client.start_as_current_span(name="smoke-test") as span:
    span.update(input="hello", output="world")

client.flush()

Run that, check the Langfuse dashboard, and you should see one trace with one span. If it's empty, double check the host and keys before building anything more complex on top.

Instrumenting a real agent loop with nested spans

Here's a minimal agent loop: it takes a user query, decides whether to call a search tool, calls it if needed, and generates a final answer. We'll wrap the whole thing in a root span and give each internal step its own child span.

from langfuse import Langfuse, observe
import anthropic

client = Langfuse()
llm = anthropic.Anthropic()

def search_tool(query: str) -> str:
    # stand-in for a real retrieval call
    return f"Top result for '{query}': agent observability requires tracing every step."

@observe(name="plan_step")
def plan(user_query: str) -> dict:
    response = llm.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=200,
        messages=[{
            "role": "user",
            "content": f"Does answering '{user_query}' require a web search? Reply with just 'yes' or 'no'."
        }]
    )
    decision = response.content[0].text.strip().lower()
    return {"needs_search": decision.startswith("y")}

@observe(name="search_step")
def run_search(user_query: str) -> str:
    return search_tool(user_query)

@observe(name="answer_step")
def answer(user_query: str, context: str | None) -> str:
    prompt = user_query if context is None else f"Context: {context}\n\nQuestion: {user_query}"
    response = llm.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=500,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

@observe(name="agent_run")
def run_agent(user_query: str) -> str:
    plan_result = plan(user_query)
    context = run_search(user_query) if plan_result["needs_search"] else None
    return answer(user_query, context)

if __name__ == "__main__":
    result = run_agent("What does agent observability with Langfuse actually track?")
    print(result)
    client.flush()

The @observe decorator does the heavy lifting: it creates a span named after the function, times it automatically, captures the function's arguments as input and its return value as output, and nests it under whatever span is currently active. Because run_agent calls plan, run_search, and answer while itself wrapped in @observe, Langfuse builds the parent-child tree without you manually threading a trace ID through every function signature.

Open the trace in the dashboard and you'll see agent_run as the root, with plan_step, search_step, and answer_step as children in execution order, each with its own latency and (for the two LLM-backed steps) token counts if you're using one of Langfuse's native integrations.

Capturing token usage and cost per step

@observe captures input/output automatically, but token usage and cost need the LLM call itself to be visible to Langfuse. There are two ways to get this:

Option 1: Use the Anthropic or OpenAI wrapper. Langfuse ships drop-in wrappers that patch the client so every call is captured with full usage data, no manual span work needed.

from langfuse.openai import openai  # drop-in replacement for the openai package

client = openai.OpenAI()

response = client.chat.completions.create(
    model="gpt-5.1",
    messages=[{"role": "user", "content": "Summarize agent observability in one sentence."}]
)

Every call through this wrapped client automatically becomes a Langfuse generation with prompt, completion, token counts, and cost, attached to whatever parent span is active.

Option 2: Log generations manually inside a span, which is what you need for providers without a wrapper, or when you want full control over what gets recorded:

from langfuse import Langfuse

client = Langfuse()

with client.start_as_current_generation(
    name="answer_step",
    model="claude-sonnet-4-5",
    input=[{"role": "user", "content": "What is agent observability?"}],
) as generation:
    response = llm.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=300,
        messages=[{"role": "user", "content": "What is agent observability?"}]
    )
    generation.update(
        output=response.content[0].text,
        usage_details={
            "input": response.usage.input_tokens,
            "output": response.usage.output_tokens,
        },
    )

start_as_current_generation is a specialized span type in Langfuse that the dashboard treats differently from a plain span: it shows up with model name, token breakdown, and a computed cost based on that model's pricing, so your cost dashboards populate without extra config.

For an agent with tool calls, wrap each tool invocation in its own regular span (not a generation, since no model call happens there) so you can see how much wall-clock time is going to tools versus the model:

@observe(name="tool_call", as_type="span")
def call_tool(tool_name: str, args: dict):
    with client.start_as_current_span(name=f"tool:{tool_name}") as span:
        span.update(input=args)
        result = TOOL_REGISTRY[tool_name](**args)
        span.update(output=result)
        return result

Grouping multi-turn conversations into sessions

A single trace covers one agent run. A real product has users coming back across multiple turns, and you want to see the whole conversation as a unit, not fifteen disconnected traces. Langfuse handles this with session_id.

import uuid

session_id = str(uuid.uuid4())  # generate once per conversation, persist it client-side

@observe(name="agent_run")
def run_agent_turn(user_query: str, session_id: str) -> str:
    client.update_current_trace(session_id=session_id, user_id="user_8842")
    plan_result = plan(user_query)
    context = run_search(user_query) if plan_result["needs_search"] else None
    return answer(user_query, context)

Every trace tagged with the same session_id groups together in the Langfuse UI as a single session view, letting you scroll through turn one, turn two, turn three in order. Attaching user_id alongside it means you can also filter "show me every session for this specific user," which is the fastest way to reproduce a bug a specific customer reported.

Scoring outputs: automated evals and human feedback

Traces tell you what happened. Scores tell you whether what happened was good. Langfuse treats scores as first-class objects attached to a trace or a specific span, and they can come from three sources.

Human feedback, captured from your product UI (a thumbs up/down button):

client.create_score(
    trace_id=trace_id,
    name="user_feedback",
    value=1,  # 1 = thumbs up, 0 = thumbs down
    data_type="NUMERIC",
)

LLM-as-judge scoring, run as a background job against completed traces, where a separate model evaluates the agent's output for correctness, helpfulness, or groundedness:

def judge_and_score(trace_id: str, question: str, answer_text: str):
    judge_response = llm.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=10,
        messages=[{
            "role": "user",
            "content": f"Question: {question}\nAnswer: {answer_text}\n\nRate the answer's factual groundedness from 0 to 1. Reply with just the number."
        }]
    )
    score_value = float(judge_response.content[0].text.strip())
    client.create_score(
        trace_id=trace_id,
        name="groundedness",
        value=score_value,
        data_type="NUMERIC",
    )

Rule-based scoring, which is cheap and deterministic, useful for things like "did the agent's output contain valid JSON" or "was the tool call schema valid":

import json

def score_json_validity(trace_id: str, output_text: str):
    try:
        json.loads(output_text)
        is_valid = 1
    except (json.JSONDecodeError, TypeError):
        is_valid = 0
    client.create_score(trace_id=trace_id, name="valid_json", value=is_valid, data_type="BOOLEAN")

Once scores accumulate, Langfuse's dashboard lets you slice them by date, by model version, or by a specific prompt version, so "did my last prompt change actually improve groundedness scores" becomes a chart instead of a guess.

Debugging a failed agent run

This is the payoff. When a user reports a bad answer, search Langfuse by user_id or session_id, open the trace, and walk the tree top to bottom. You'll typically find one of these patterns:

1. A tool call span with an empty or error output, meaning the agent silently swallowed a failure and answered anyway. Fix: make the answer step check tool outputs for error markers before synthesizing a response.
2. A generation span where the input prompt is missing context you expected to be there, meaning your retrieval step ran but the plan step decided not to call it. Fix: tighten the planning prompt or lower the threshold for triggering search.
3. A trace with ten nested `tool_call` spans all calling the same tool with near-identical arguments, meaning the agent is looping. Fix: add a max-iteration guard and log a warning score when it triggers.
4. A generation span with normal output but 12,000 input tokens, meaning context is bloating unnoticed (stale conversation history, oversized retrieval chunks). Fix: cap context size and add token-count alerting.

None of these are visible from a flat log file. They're visible in about ten seconds once you have the trace tree open, because the shape of the tree itself tells you where things diverged.

Setting up alerts and dashboards for production agents

Once tracing is in place, treat Langfuse like any other production monitoring surface, not just a debugging tool you open reactively.

• Track p50/p95 latency per span type. If search_step p95 latency creeps from 400ms to 4s, that's your retrieval backend degrading, and you'll see it before users complain.
• Track average score trends per prompt version. Tag traces with a prompt_version metadata field on every deploy, and you can directly compare groundedness or user feedback scores across versions to know if a prompt change helped or hurt.
• Track token cost by session and by user. A single runaway agent loop can burn through a surprising amount of spend before anyone notices. Cost-per-session dashboards catch that early.
• Export low-scoring traces into a dataset. Langfuse lets you curate traces (especially the low-scoring or flagged ones) into a dataset you can replay against a new prompt or model version before shipping it, turning your worst production failures into a regression test suite.

client.create_dataset_item(
    dataset_name="low_groundedness_cases",
    input={"question": question},
    expected_output=None,  # to be filled in by a human reviewer
    metadata={"original_trace_id": trace_id},
)

That last pattern closes the loop: production failures become eval cases, eval cases catch regressions before the next deploy, and the whole cycle runs without anyone manually copy-pasting bad conversations into a spreadsheet.

FAQ

Does Langfuse work with any LLM provider, or only OpenAI and Anthropic? Langfuse is provider-agnostic. It ships convenience wrappers for the most common SDKs (OpenAI, Anthropic, and others), but the underlying start_as_current_span and start_as_current_generation APIs accept input, output, and usage data manually, so any provider, including a self-hosted open source model, works the same way.

Do I need to self-host Langfuse, or is the cloud version fine for production agents? Both are production-grade. The managed cloud is the faster path to get started and removes ops overhead. Self-hosting matters when you have strict data residency requirements or want traces to never leave your own infrastructure, since agent traces often contain full user conversations and tool arguments.

How is this different from just adding print statements or a logging library? Print statements and flat logs don't capture the parent-child structure of an agent's execution, don't attach cost or token usage to individual steps, and don't give you a queryable UI to search across thousands of runs by user, session, or score. Agent observability with Langfuse gives you the tree structure and the query layer on top of the raw data, which is what makes debugging a specific bad run fast instead of a manual log-grepping exercise.

Will adding Langfuse instrumentation slow down my agent? The SDK batches and flushes trace data asynchronously in a background thread, so it doesn't block your agent's critical path. Call client.flush() explicitly at the end of short-lived scripts (like serverless functions) to make sure buffered data is sent before the process exits; long-running servers can rely on the background flush interval.

Can I use Langfuse to compare two different prompt versions or two different models head to head? Yes. Tag traces with metadata like prompt_version or model_name at generation time, then filter and compare score distributions, latency, and cost between tags directly in the dashboard. This is the standard way to validate that a prompt or model swap is actually an improvement before rolling it out to all traffic.

What's the difference between a span and a generation in Langfuse? A span is a generic unit of work with timing, input, and output, used for things like tool calls or planning steps. A generation is a specialized span specifically for LLM calls; it additionally carries model name, token usage, and cost, which the dashboard uses to compute per-model spend and latency breakdowns that a plain span doesn't support.