LangChain Callbacks and Tracing: A Practical Guide for Debugging LLM Chains

LangChain callbacks are the hook system that lets you observe what happens inside a chain, agent, or LLM call as it runs, and tracing is what you get when you wire those callbacks into a system that records the full execution tree so you can replay and debug it later. If you have ever wondered why an agent picked a certain tool, why a chain returned an empty string, or where your token budget went, callbacks and tracing are the tools that answer those questions. This guide walks through the callback system from first principles, then shows how to wire up tracing for real debugging work.

Most engineers meet callbacks the hard way: a chain fails in production, the only evidence is a stack trace pointing at a retry wrapper, and there is no record of what the model actually saw or returned. Callbacks fix that by giving you a structured event stream for every run. Tracing turns that stream into something you can search, diff, and share with a teammate.

What LangChain callbacks actually are

A callback in LangChain is an object that implements a set of handler methods, each corresponding to an event in the lifecycle of a run: a chain starting, an LLM being called, a tool being invoked, an agent taking an action, an error being raised, and so on. LangChain calls these methods automatically as your code executes, whether that code is a single ChatOpenAI call or a multi-step agent built with LangGraph.

The base class you extend is BaseCallbackHandler. It exposes methods like:

• on_llm_start / on_llm_end / on_llm_error
• on_chain_start / on_chain_end / on_chain_error
• on_tool_start / on_tool_end / on_tool_error
• on_agent_action / on_agent_finish
• on_text
• on_retry

You do not have to implement all of them. Override only the ones you care about, and LangChain will call the rest as no-ops.

Here is a minimal handler that logs every LLM call with timing:

import time
from langchain_core.callbacks import BaseCallbackHandler

class TimingHandler(BaseCallbackHandler):
    def on_llm_start(self, serialized, prompts, **kwargs):
        self._start = time.monotonic()

    def on_llm_end(self, response, **kwargs):
        elapsed = time.monotonic() - self._start
        print(f"LLM call took {elapsed:.2f}s")

    def on_llm_error(self, error, **kwargs):
        print(f"LLM call failed: {error}")

You attach it at call time by passing config={"callbacks": [TimingHandler()]}, or at construction time when building a chain. Passing it per-call is usually better because it avoids leaking handler state across concurrent requests.

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")
response = llm.invoke(
    "Summarize the plot of Dune in two sentences.",
    config={"callbacks": [TimingHandler()]},
)

Where callbacks fit: constructor vs request-time

LangChain supports attaching callbacks in two places, and the distinction matters for anything beyond a toy script.

Constructor callbacks are set when you build an object, for example ChatOpenAI(callbacks=[handler]). They apply to every call made through that object for its entire lifetime. This is convenient for global logging but dangerous in a server process handling multiple concurrent requests, because the handler instance is shared and any per-request state you store on it (like self._start above) will race across requests.

Request-time callbacks are passed through the config argument on invoke, stream, batch, or ainvoke. LangChain propagates this config down through every nested chain, tool, and sub-call automatically, using Python's contextvars under the hood. This is the pattern to use in any multi-user or async application.

async def handle_request(user_input: str, request_id: str):
    handler = RequestLogger(request_id=request_id)
    result = await agent.ainvoke(
        {"input": user_input},
        config={"callbacks": [handler], "run_name": f"req-{request_id}"},
    )
    return result

Because the handler is created fresh per request, there is no shared mutable state, and each run's events stay isolated even under concurrent load.

Streaming tokens with callbacks

The most common reason people reach for callbacks is streaming. on_llm_new_token fires once per token (or token chunk, depending on the provider) as the model generates output, which is what powers typewriter-style UIs.

class StreamPrinter(BaseCallbackHandler):
    def on_llm_new_token(self, token, **kwargs):
        print(token, end="", flush=True)

llm = ChatOpenAI(model="gpt-4o-mini", streaming=True)
llm.invoke("Write a haiku about databases.", config={"callbacks": [StreamPrinter()]})

Note the streaming=True flag on the model constructor. Without it, some providers buffer the full response and call on_llm_new_token all at once, or not at all, even if a callback is attached. If you are building a chat UI, prefer the higher-level .stream() or .astream() methods over manually catching tokens in a callback; they give you an iterator directly and handle the plumbing for you. Reach for on_llm_new_token when you need to do something else simultaneously, like pushing tokens onto a websocket while also accumulating them for a database write.

Tracing agent decisions

Agents are where callbacks earn their keep, because an agent's behavior is not a single LLM call, it is a loop: think, pick a tool, observe the result, think again. Without visibility into that loop, debugging an agent that picked the wrong tool or looped forever is close to guesswork.

on_agent_action fires every time the agent decides to call a tool, giving you the tool name, the input it constructed, and the raw log text the model produced to justify the choice. on_tool_start and on_tool_end bracket the actual tool execution. on_agent_finish fires once the agent produces a final answer instead of another tool call.

class AgentTracer(BaseCallbackHandler):
    def on_agent_action(self, action, **kwargs):
        print(f"[agent] calling tool={action.tool!r} input={action.tool_input!r}")

    def on_tool_end(self, output, **kwargs):
        print(f"[tool] returned: {str(output)[:200]}")

    def on_agent_finish(self, finish, **kwargs):
        print(f"[agent] final answer: {finish.return_values}")

Attach this to an agent executor or a LangGraph run and you get a readable trace of every decision without touching the agent's internal logic. This is usually the fastest way to answer "why did the agent do that" questions, faster than reading through prompt templates trying to reason about what the model probably saw.

Handling errors and retries

on_llm_error, on_chain_error, and on_tool_error fire when something throws inside a run. on_retry fires when LangChain's built-in retry logic (for example on a RunnableRetry wrapper, or provider-level rate limit handling) kicks in before an error is raised to the caller.

A pattern worth adopting: route errors from callbacks into your existing structured logging or error-tracking tool rather than printing them. The callback gives you the run ID and parent run ID, which lets you correlate an error with the exact chain of calls that produced it.

class ErrorReporter(BaseCallbackHandler):
    def on_tool_error(self, error, *, run_id, parent_run_id, **kwargs):
        report_error(
            tool="unknown",
            error=str(error),
            run_id=str(run_id),
            parent_run_id=str(parent_run_id) if parent_run_id else None,
        )

report_error here is a stand-in for whatever your team already uses, a logging call, a Sentry capture, or a write to an internal events table.

Async callbacks

Everything above uses the synchronous BaseCallbackHandler. If your application is async, for example an agent served through FastAPI with ainvoke, use AsyncCallbackHandler instead and define the same methods as coroutines:

from langchain_core.callbacks import AsyncCallbackHandler

class AsyncLogger(AsyncCallbackHandler):
    async def on_llm_end(self, response, **kwargs):
        await write_log_async(response)

Mixing sync callback handlers into an async run generally still works because LangChain runs sync handlers in a thread pool, but if a sync handler does blocking I/O (a synchronous database write, for instance), it will tie up a thread pool worker on every call. For anything doing real I/O inside a callback, write the async version.

Built-in tracing with LangSmith

Custom BaseCallbackHandler classes are great for one-off logging, but for production observability, most teams should use the built-in tracing integration rather than hand-rolling one. LangChain ships first-party support for LangSmith, Anthropic's and other vendors' equivalents follow the same OpenTelemetry-adjacent pattern, and the mechanism is the same across all of them: a small set of environment variables turns on a global tracer that captures every run in your process without touching application code.

export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=your-api-key
export LANGCHAIN_PROJECT=my-agent-project

With those set, every invoke, stream, and batch call across your app gets traced automatically: full inputs and outputs at every step, token counts, latency per node, and the complete tool-call tree for agents. This is the practical alternative to writing your own AgentTracer class for anything beyond local debugging, because it gives you a searchable UI, run comparison, and the ability to share a specific trace URL with a teammate instead of pasting console output into Slack.

You can combine both approaches. Keep the tracing environment variables on for the always-available trace history, and add a targeted custom callback when you need something the trace UI does not surface, like a business metric derived from the tool's output.

Tagging and naming runs for searchable traces

A trace with a thousand identical-looking "ChatOpenAI" entries is not useful. Use tags and run_name in the config to make traces searchable later:

result = chain.invoke(
    {"question": user_question},
    config={
        "tags": ["support-bot", "tier-1"],
        "run_name": "support-triage",
        "metadata": {"user_id": user_id, "session_id": session_id},
    },
)

Tags and metadata propagate to every nested run inside the call, so a single top-level tag lets you filter an entire agent's execution tree in your tracing tool later, including every LLM call and tool invocation nested inside it. This is the single highest-leverage habit for teams debugging agents in production: tag by feature, by user tier, or by experiment variant, and you turn a wall of traces into something you can actually query.

Getting run IDs back for correlation

Every callback method receives run_id and parent_run_id as keyword arguments. Store the top-level run_id alongside your own request ID so that when a user reports a bad answer, you can go straight from your application logs to the exact trace.

from langchain_core.tracers.context import collect_runs

with collect_runs() as cb:
    result = agent.invoke({"input": user_input})
    run_id = cb.traced_runs[0].id

save_to_request_log(request_id=my_request_id, langchain_run_id=str(run_id))

collect_runs is a context manager built specifically for capturing the run ID without writing a full custom handler, useful when all you need is the correlation ID, not a stream of intermediate events.

Callbacks vs middleware vs LangGraph node hooks

If you are building with LangGraph rather than the older AgentExecutor, callbacks still work exactly the same way, because LangGraph is built on the same Runnable and callback infrastructure as the rest of LangChain. Every node in a graph is a Runnable, so on_chain_start and on_chain_end fire for each node, and you get the same tracing behavior with LANGCHAIN_TRACING_V2 turned on, now visualized as a graph in the tracing UI instead of a flat call stack. This matters if you are migrating from a classic agent executor to LangGraph: your existing custom callback handlers keep working without modification, and your tracing setup does not need to change at all.

A debugging checklist

When a chain or agent misbehaves, work through callbacks in this order rather than reaching for print statements scattered through the source:

1. Turn on LANGCHAIN_TRACING_V2 and reproduce the failure. Read the trace tree first; it usually tells you exactly which step returned something unexpected.
2. Check on_llm_start inputs to confirm the prompt the model actually received matches what you expect. Prompt template bugs are the most common cause of "the model did something weird."
3. Check on_tool_start inputs and on_tool_end outputs for agents. A tool returning malformed JSON or an empty string is a frequent, easy-to-miss cause of agent loops.
4. If the trace UI does not show enough detail, add a targeted custom handler for the specific event you need, rather than a broad one that logs everything.
5. Confirm callbacks are attached at request time, not constructor time, if you are running concurrent requests and seeing cross-contaminated logs.

FAQ

Do LangChain callbacks slow down my application? A lightweight handler that just logs or appends to a list adds negligible overhead. Handlers that do blocking I/O, like a synchronous network call inside on_llm_end, can add real latency because they run inline with the request unless you offload the work to a background task or queue.

Can I attach a callback to only part of a chain? Yes. Pass the callback in the config for a specific invoke call on a sub-chain, or attach it directly to one Runnable using .with_config(callbacks=[handler]) before composing it into a larger chain. Callbacks passed at the top level propagate to every nested step, but a callback attached to a single Runnable only fires for that Runnable.

What is the difference between `callbacks` in the constructor and in `config`? Constructor callbacks live for the object's lifetime and apply to every call made through it; they are convenient for scripts but risky for concurrent servers because handler state is shared. Config (request-time) callbacks are scoped to a single call and propagate through contextvars, which is the safer pattern for anything serving multiple users at once.

Do I need LangSmith to use tracing, or can I self-host it? LangSmith is the first-party option and the one with the least setup, but the callback system itself is vendor-neutral. You can write a custom BaseCallbackHandler that ships events to any backend, an internal logging pipeline, an OpenTelemetry collector, or a plain database table, and get equivalent tracing without depending on a specific vendor.

Why isn't `on_llm_new_token` firing for my model? Check three things: the model was constructed with streaming=True, the callback is attached at the same level as the call (not on an unrelated object), and the underlying provider actually supports token-level streaming for the model you selected. Some providers batch partial output server-side, which shows up as one large token event instead of many small ones.

Can callbacks modify the input or output of a chain? No. Callback handlers are observers; the methods return None and LangChain does not use their return values to alter execution. If you need to transform inputs or outputs, do that in the chain itself with a RunnableLambda, not in a callback. Use callbacks for observation and side effects like logging, not for changing behavior.