LangSmith for Agent Debugging: Tracing Multi-Step Decisions

Your agent worked perfectly in the demo. Then a real user asked it a slightly different question, it called the wrong tool with the wrong arguments, looped three times, burned twelve thousand tokens, and confidently returned an answer that was wrong. You open your terminal logs and find... a wall of JSON fragments that tells you almost nothing about *why* the agent decided what it decided. This is the moment every AI engineer hits, and it is exactly the problem LangSmith was built to solve. LangSmith agent debugging is not about reading logs harder. It is about capturing the full decision tree of an agent run — every LLM call, every tool invocation, every intermediate state — as a structured, navigable trace, so you can walk through the agent's reasoning the way you would step through code in a debugger. In this guide, we will set up tracing from scratch, trace a real multi-step agent, learn to read the run tree, and build a repeatable debugging workflow that scales from local development to production.

Why Agent Debugging Is Nothing Like Normal Debugging

When a regular function fails, you have a stack trace, a line number, and deterministic behavior. Run it again with the same input and it fails the same way. Agents break all three of these assumptions at once.

First, agents are non-deterministic. The same prompt can produce a different tool-call sequence on the next run, because the LLM's sampling introduces variance at every decision point. A bug you saw once may not reproduce on demand, which means the trace of the *original failing run* is often the only evidence you will ever have. If you were not recording traces when the failure happened, that evidence is gone.

Second, agent failures are usually not exceptions. The code runs fine. The HTTP calls return 200. What fails is a *decision*: the agent picked the search tool when it should have queried the database, or it stopped after one retrieval when the question needed three, or it paraphrased a tool result and dropped the one number that mattered. Nothing throws, so nothing shows up in error monitoring. These are semantic failures, and they are invisible to traditional observability tools built around exceptions and status codes.

Third, the failure is often several steps removed from the symptom. The final answer is wrong because step 5 summarized badly, because step 3 retrieved the wrong document, because step 1 rewrote the user's query too aggressively. To debug that chain you need to see every step's exact inputs and outputs, in order, with the ability to drill into any one of them. That is precisely what a LangSmith trace gives you: a hierarchical record of the entire run where each node — an LLM call, a tool execution, a retriever query, a sub-chain — is captured with its full input, output, latency, and token usage.

The mental shift is this: with agents, observability is not a production nice-to-have you add later. It is your primary debugging tool from day one, because reading traces *is* how you debug an agent.

The Core Model: Runs, Traces, and the Run Tree

Before touching code, it helps to internalize LangSmith's data model, because everything in the UI and the SDK maps back to three concepts.

A run is a single unit of work: one LLM call, one tool execution, one function invocation. Every run records its inputs, outputs, start and end time, error state (if any), and metadata such as token counts and model name.

A trace is the full tree of runs produced by one end-to-end execution — for an agent, that is everything that happened between the user's message and the final answer. The top-level run is the root; every LLM call and tool call the agent made appears as a child or deeper descendant.

The run tree is the hierarchy itself. A typical ReAct-style agent trace looks like this: a root run for the agent, under it an alternating sequence of LLM runs (where the model decides what to do) and tool runs (where the decision is executed), and finally an LLM run that produces the answer. In a LangGraph application, each graph node shows up as its own child run, so the trace mirrors your graph topology exactly.

This structure is what makes multi-step debugging tractable. When an agent takes nine steps, you do not scroll through nine interleaved log statements — you expand a tree, see at a glance which step was slow, which step errored, and which step's output looks suspicious, then click into that one node and inspect the exact prompt the model saw at that moment. The "exact prompt the model saw" part is the killer feature: agent bugs are overwhelmingly caused by a mismatch between what you *think* the model received and what it *actually* received — truncated context, a malformed tool schema, a system prompt that got overwritten, message history assembled in the wrong order. The trace shows you the raw truth.

Setting Up Tracing in Five Minutes

LangSmith tracing is designed to be nearly zero-effort if you use LangChain or LangGraph, and only slightly more effort if you do not. Everything starts with environment variables.

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="your-api-key"
export LANGSMITH_PROJECT="agent-debugging-demo"

With those set, any LangChain or LangGraph code in the process is traced automatically — no code changes at all. Every chain invocation, model call, and tool execution flows into the project you named. Projects are simply named buckets of traces; use one per application per environment (something like support-agent-dev and support-agent-prod) so production noise never drowns out your local experiments.

If your agent is plain Python — direct OpenAI or Anthropic SDK calls, custom orchestration, no framework — you use the @traceable decorator to declare the units of work yourself:

from langsmith import traceable
from langsmith.wrappers import wrap_openai
from openai import OpenAI

client = wrap_openai(OpenAI())  # auto-traces every completion call

@traceable(run_type="tool")
def search_orders(customer_id: str, status: str) -> list:
    """Look up orders for a customer, filtered by status."""
    return db.query_orders(customer_id, status)

@traceable(run_type="chain", name="support_agent")
def run_agent(user_message: str) -> str:
    messages = [{"role": "system", "content": SYSTEM_PROMPT},
                {"role": "user", "content": user_message}]
    for _ in range(MAX_STEPS):
        response = client.chat.completions.create(
            model="gpt-4o", messages=messages, tools=TOOL_SCHEMAS
        )
        choice = response.choices[0]
        if not choice.message.tool_calls:
            return choice.message.content
        for call in choice.message.tool_calls:
            result = dispatch_tool(call)   # also decorated with @traceable
            messages.append(tool_result_message(call, result))
    return "Agent hit max steps without an answer."

The nesting happens automatically: because search_orders is called inside run_agent, its run appears as a child of the agent's run in the trace. The wrap_openai wrapper captures every raw model call, including the full message array and the tool schemas sent with it. The result is the same navigable run tree you would get from LangGraph, built from your own functions.

Two setup habits will pay off immediately. Give runs meaningful names (name="support_agent" beats the default function name when you are scanning hundreds of traces), and set run_type accurately (llm, tool, chain, retriever) because the UI renders and filters each type differently.

Tracing a Multi-Step LangGraph Agent End to End

Let us make this concrete with a LangGraph agent that has real multi-step behavior: a research assistant that can search, fetch documents, and calculate, and must chain those tools to answer compound questions.

from langgraph.prebuilt import create_react_agent
from langchain_core.tools import tool

@tool
def search_web(query: str) -> str:
    """Search the web and return the top results as text."""
    return web_search_client.search(query, max_results=5)

@tool
def fetch_page(url: str) -> str:
    """Fetch a URL and return its main text content."""
    return page_loader.get_clean_text(url)

@tool
def calculate(expression: str) -> str:
    """Evaluate a math expression, e.g. '(1420 - 1180) / 1180 * 100'."""
    return str(safe_eval(expression))

agent = create_react_agent(
    model="anthropic:claude-sonnet-4-5",
    tools=[search_web, fetch_page, calculate],
    prompt="You are a careful research assistant. Verify numbers "
           "against sources before calculating with them.",
)

result = agent.invoke(
    {"messages": [("user", "How much did the framework's weekly downloads "
                           "grow between its last two major releases, in percent?")]},
    config={"metadata": {"user_tier": "beta"}, "tags": ["research-flow"]},
)

Run this with tracing enabled and open the trace in LangSmith. What you see is the agent's entire thought process laid out as a tree. The root run holds the final answer and total cost. Under it, the first LLM call shows the model reading the question and deciding to call search_web — you can see the exact tool schema it was given and the exact query string it generated. Then the search_web tool run shows the raw results that came back. Then another LLM call where the model reads those results and decides it needs fetch_page for a specific URL. Then the fetch. Then a calculate call with the expression it built from the numbers it extracted. Then the final LLM call composing the answer.

Every decision point is inspectable. If the final percentage is wrong, you walk the tree and find where reality diverged from intent. Maybe the search results never contained download numbers and the model hallucinated them — visible instantly by comparing the tool output against the next LLM call's reasoning. Maybe the model extracted the right numbers but built the wrong expression — visible in the calculate run's input. Maybe everything was right until the final summarization rounded aggressively. In a log file, distinguishing these three root causes is an afternoon of archaeology. In a trace, it is ninety seconds of clicking.

The metadata and tags in the config are not decoration. They are attached to the trace and become filterable dimensions later — the difference between "search all traces" and "show me research-flow traces from beta users that errored," which is a query you will run constantly once real traffic arrives.

Reading a Trace Like a Debugger: A Repeatable Workflow

Opening a trace with forty nested runs can feel overwhelming, so it helps to have a fixed reading order. Here is the workflow I teach, and it works for nearly every agent bug.

1. Read the root run first. Compare the original user input against the final output. Classify the failure: wrong answer, incomplete answer, refused to answer, took too long, or cost too much. This determines what you look for next.
2. Scan the tree shape before reading any content. Count the steps. An agent that should take three steps but took eleven has a looping or planning problem regardless of what the steps contain. An agent that took one step when the task needed four gave up early — usually a prompt problem, not a tool problem.
3. Find the first bad step, not the last one. Walk top-down and check each LLM run's decision against what a competent human would have decided given the same visible context. The first divergence is your root cause; everything after it is contamination.
4. At the suspect step, read the rendered prompt in full. Not your prompt template — the actual assembled messages the model received. Check that the system prompt survived, the history is in order, the tool results were actually inserted, and nothing was truncated. A shocking fraction of "the model is dumb" bugs turn out to be "the model never saw the data."
5. Check the tool runs bracketing the bad decision. Was the tool's output empty, malformed, or subtly different in shape from what your prompt promised the model it would get? Models degrade fast when tool outputs do not match the schema described to them.
6. Only then blame the model. If the context was complete and correct and the decision was still bad, now you have a genuine reasoning failure — which you fix with a better prompt, few-shot examples of the correct decision, a stronger model for that step, or by restructuring the choice so the model has fewer ways to be wrong.

The playground integration makes step 6 fast: from any LLM run in a trace you can open that exact call — same messages, same tools, same parameters — in an editable playground, tweak the prompt, and re-run it against the captured context. You are iterating on the precise failing state rather than trying to reconstruct it, which turns prompt debugging from guesswork into something much closer to test-driven development.

The Four Failure Patterns You Will See Over and Over

After enough traces, you start recognizing failure signatures on sight. Four patterns account for most multi-step agent bugs.

Wrong tool, or wrong arguments. The trace shows a reasonable question followed by a tool call that does not fit it — search_web when the answer was in the database, or a date argument formatted as 03/04/2026 when the tool wanted ISO. The fix is almost always in the tool's schema: sharpen the description, rename ambiguous parameters, add explicit format examples in the docstring. The model chooses tools by reading those descriptions, and vague descriptions produce vague choices.

The loop. The tree shows the same tool called repeatedly with near-identical arguments, results ignored each time. This usually means the tool's output does not tell the model whether it succeeded — an empty list that the model cannot distinguish from a failed call, so it retries forever. Make tools return explicit signals ("No orders found for this customer; do not retry with the same ID") and enforce a step budget so loops fail loudly instead of silently draining tokens.

Context poisoning. An early step injects something wrong — a hallucinated fact, an error message pasted into the history, a stale retrieval — and every subsequent step reasons correctly from the poisoned premise. The signature in the trace is a run of individually sensible decisions built on one bad input. The fix belongs at the injection point: validate tool outputs before appending them, or add a verification step after retrieval.

Silent truncation. Long agent runs accumulate huge message histories, and something — your trimming logic, a context-window limit, a summarization step — drops content mid-run. The model then "forgets" instructions or facts it clearly had earlier. In the trace, you catch this by diffing the message list between consecutive LLM runs and spotting where the earlier content vanished. Token counts per run, visible right in the tree, make the shrinkage easy to see.

Naming the pattern matters because each has a different fix, and the trace tells you which one you have before you change a single line.

Filtering and Searching Traces When You Have Thousands

Local debugging is one trace at a time. Real debugging — the kind you do after launch — is finding the ten relevant traces among fifty thousand. This is where the metadata and tags you attached earlier become load-bearing.

LangSmith's trace view supports filtering on error status, latency thresholds, token counts, run names, tags, metadata keys, and full-text search over inputs and outputs. In practice, a few queries do most of the work: all errored traces in the last day; all traces slower than some latency budget; all traces where a particular tool was called; all traces containing a specific phrase a user reported. You can combine filters with tree-aware conditions — for example, root runs whose *subtree* contains a failed tool call — which is how you find "conversations where the database lookup failed but the agent answered anyway," a class of bug that is invisible at the root level.

To make this work, be disciplined about attaching identifiers at invocation time:

result = agent.invoke(
    {"messages": [("user", question)]},
    config={
        "metadata": {
            "session_id": session_id,
            "user_id": user_id,
            "app_version": APP_VERSION,
            "prompt_version": "support-v14",
        },
        "tags": ["production", "support-agent"],
    },
)

The prompt_version field deserves special mention. When you ship a prompt change, tagging traces with the version lets you compare failure rates and behavior before and after — filter to support-v13, then support-v14, and look at the same slice of traffic. Without a version tag, prompt regressions are nearly impossible to attribute. The session_id field, meanwhile, lets you group a multi-turn conversation into a single thread view, so you can debug failures that only emerge across turns — the agent forgetting something from three messages ago is only diagnosable when you can see all three messages' traces together.

From Debugging to Regression Testing: Datasets and Evals

Here is the workflow that separates teams who debug agents well from teams who debug the same bug every month: every interesting trace becomes a test case.

When you find a failing trace and fix the cause, do not just move on. Add that trace's input — and the output you *wanted* — to a LangSmith dataset. One click in the UI ("Add to dataset") or a small SDK call does it. Over weeks, this builds a regression suite made entirely of real failures, which is far more valuable than any synthetic test set because it encodes the actual ways your users break your agent.

Then, before shipping any change — a new prompt, a swapped model, a refactored tool — run the agent against the dataset:

from langsmith import Client

client = Client()

def correct_tool_sequence(outputs: dict, reference_outputs: dict) -> bool:
    """Did the agent call the tools the reference run says it should?"""
    called = [c["name"] for c in outputs.get("tool_calls", [])]
    return called == reference_outputs["expected_tools"]

results = client.evaluate(
    lambda inputs: run_agent(inputs["question"]),
    data="support-agent-regressions",
    evaluators=[correct_tool_sequence],
    experiment_prefix="support-v15-candidate",
)

Each evaluation run is itself fully traced, so when a case fails you do not just get a red X — you get the complete trace of the failing attempt, ready for the same debugging workflow as before. You can also compare two experiments side by side and see exactly which cases regressed between prompt v14 and v15. Notice that the evaluator above checks the *trajectory* (which tools were called, in what order), not just the final answer. For agents, trajectory evals catch a whole class of problems that answer-checking misses: right answer by luck, wrong answer with the right process, or correct output achieved with triple the necessary tool calls.

For fuzzier criteria — "is this answer grounded in the retrieved documents?" — LLM-as-judge evaluators slot into the same evaluators list. The loop stays identical: trace, debug, fix, add to dataset, evaluate every future change against it.

Debugging in Production: Monitoring and Feedback

Production changes the game in one important way: you no longer know which runs failed. Users do not file bug reports for most bad answers; they just leave. So production debugging starts with instrumenting *signals* that surface the traces worth reading.

The first signal is user feedback. Wire your thumbs-up/thumbs-down (or "was this helpful?") UI to LangSmith's feedback API, attaching a score to the trace's run ID. Now "show me all thumbs-down traces from yesterday" is a saved filter, and your morning debugging session starts with a pre-sorted queue of confirmed failures, each with its complete decision tree already captured.

The second signal is automated. You can configure online evaluators that run on a sample of production traces as they arrive — an LLM judge scoring answer groundedness, a rule flagging traces where the agent exceeded a step budget, a check for tool errors that did not surface to the user. These annotate traces with scores you can filter and alert on, catching quality drift that no exception monitor would ever see.

The third is the boring-but-critical operational layer: dashboards on latency, token cost per trace, error rates, and tool-failure rates over time. Agents have a distinctive failure mode where behavior degrades without any deploy — an upstream API slows down, a model provider updates something, traffic shifts toward a query type your prompt handles badly. Trend lines catch these; individual traces then explain them.

A note on cost and privacy, since both come up immediately in production: sample your tracing if volume is high (trace every run in development, a percentage in production, but always trace runs that error or receive negative feedback), and use LangSmith's input/output redaction hooks to strip PII before traces leave your service. Debugging power is only useful if you are allowed to keep it turned on.

Keep Learning

The pattern behind everything in this article is simple: agents fail in their decisions, not their exceptions, so you debug them by capturing and reading decisions — one trace at a time in development, filtered and prioritized at scale in production, and frozen into datasets so fixed bugs stay fixed. LangSmith gives you each layer of that stack: the run tree for stepping through a single agent's reasoning, filters and feedback for finding the needles in production, and datasets with evals for turning every failure into a permanent regression test. Set the environment variables today, and the next time your agent does something baffling, you will have the evidence in front of you instead of a wall of JSON.

If you want to go deeper — building custom evaluators, tracing complex LangGraph topologies with subgraphs and interrupts, setting up online evaluation and alerting, and running full prompt-regression pipelines in CI — the LangSmith Tutorial course on teachyou.ai walks through all of it hands-on, from your first trace to a production observability setup, with real agents that break in realistic ways so you can practice the debugging workflow yourself.