Monitoring AI Agents in Production

Agent monitoring production is a different problem than monitoring a normal web service. A regular API either returns the right status code or it doesn't. An AI agent can return HTTP 200, produce a fluent, confident response, and still be completely wrong: it called the wrong tool, hallucinated a customer ID, or looped through the same retrieval step four times before giving up. None of that shows up in a standard uptime dashboard. If you're running agents in production, you need instrumentation built for multi-step, non-deterministic workflows, not just request latency and error rate.

This article covers what to track, how to trace agent runs end to end, how to catch silent failures, and how to wire alerts that actually page a human when it matters.

Why agent monitoring production setups differ from API monitoring

A typical microservice has a small, predictable set of failure modes: 5xx errors, timeouts, elevated p99 latency. You alert on those and move on. An agent adds a layer of failure modes that live entirely inside a "successful" response:

• The agent picks the wrong tool for the task (calls search_orders when it should call refund_order).
• The agent calls the right tool with malformed or hallucinated arguments.
• The agent loops: retrieval -> reasoning -> retrieval -> reasoning, burning tokens and time without converging.
• The final answer is fluent but factually wrong, and no exception was ever thrown.
• A tool call succeeds technically (the API returns data) but the data is stale, empty, or irrelevant, and the agent doesn't notice.

Standard APM tools like generic error tracking will report all of these runs as healthy 200s. This is why agent monitoring production requires tracing the full decision chain, not just the outer HTTP request.

The four layers you need to instrument

Think of agent observability as four layers, from outermost to innermost:

1. Request layer: the user-facing call in and the final response out. Latency, success/failure, cost.
2. Trace layer: every step the agent took to get from input to output, including planning, tool calls, and intermediate reasoning.
3. Tool layer: each individual tool/function call, its arguments, its return value, and how long it took.
4. Model layer: raw model calls, including prompt, completion, token counts, and which model/version served the request.

You need all four because a problem at the tool layer (a flaky downstream API) looks completely different in your dashboards than a problem at the model layer (a prompt regression after a model version bump), but both will manifest as "response quality dropped" at the request layer if that's all you're tracking.

Setting up tracing with OpenTelemetry

Most production agent stacks in 2026 build on OpenTelemetry (OTel) because it gives you vendor-neutral traces you can send to whatever backend you already run: Datadog, Honeycomb, Grafana Tempo, or an LLM-specific tool like Langfuse or Arize Phoenix.

The key idea: each agent run is a single trace, and each step inside it (a model call, a tool call, a retrieval) is a span nested under that trace. Here's a minimal Python setup using the OTel SDK with a generic agent loop:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint="https://your-otel-collector/v1/traces"))
)
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("agent-runtime")


def run_agent(user_input, session_id):
    with tracer.start_as_current_span("agent.run") as run_span:
        run_span.set_attribute("session.id", session_id)
        run_span.set_attribute("input.text", user_input)

        plan = plan_step(user_input)
        run_span.set_attribute("agent.plan", plan.summary)

        for step in plan.steps:
            with tracer.start_as_current_span(f"agent.step.{step.name}") as step_span:
                step_span.set_attribute("step.type", step.type)
                result = execute_step(step)
                step_span.set_attribute("step.success", result.success)
                step_span.set_attribute("step.duration_ms", result.duration_ms)
                if not result.success:
                    step_span.set_attribute("step.error", str(result.error))

        final = synthesize_response(plan)
        run_span.set_attribute("output.text", final)
        return final

Wrap every tool call the same way, one span per call, with the tool name, arguments (redact anything sensitive before you log it), return value size, and latency as attributes. This gives you a waterfall view per agent run: you can open a single trace and see exactly which step took nine seconds, which tool returned an empty array, and which model call cost the most tokens.

If you're using a framework like LangChain, LlamaIndex, or the Claude Agent SDK, check for built-in OTel or callback hooks first. Most agent frameworks in 2026 expose a callback interface you can attach an exporter to instead of hand-instrumenting every call.

Metrics that actually predict incidents

Traces are for debugging one bad run. Metrics are for noticing that things are trending bad across thousands of runs. The ones worth dashboarding:

• Tool call error rate, per tool. Not an aggregate. A spike in errors from one specific tool (say, your inventory lookup) tells you exactly where to look. Aggregate error rate just tells you something is wrong, somewhere.
• Step count per run. A sudden jump in average steps-per-run usually means the agent started looping or fell into a retry storm. This is often the earliest signal of a broken run, well before latency or cost alerts fire.
• Token cost per successful task. Track cost against completed tasks, not just raw token spend. A cost spike with flat completed-task volume means the agent is burning tokens without finishing anything.
• Time to first tool call. If this creeps up, your planning prompt or context window is getting bloated, often from an unbounded conversation history or a retrieval step returning too much text.
• Fallback/escalation rate. If your agent has a "hand off to a human" or "return a generic apology" path, track how often it's triggered. A rising escalation rate is a leading indicator of quality degradation, often before users start complaining.
• Output length distribution. Sudden truncation (hitting max tokens) or unusually short responses are cheap to detect and often correlate with real failures.

Put these on a dashboard next to your normal service metrics (p50/p95/p99 latency, request volume, uptime) so on-call engineers see agent-specific health alongside infrastructure health, not in a separate tool nobody checks.

Catching silent failures with evals in the loop

The hardest category is the fluent-but-wrong response: no error, no exception, no missing field, just an answer that's incorrect. You cannot catch all of these with metrics alone. The practical approach is running lightweight evaluations on a sample of live traffic, not just in a pre-deploy test suite.

A common pattern: sample 2-5% of production runs, and for each sampled run, fire a cheap "judge" call that checks a small set of properties, not full correctness (that usually needs a human or a ground-truth dataset):

def judge_run(trace_record):
    checks = {
        "used_expected_tool": trace_record.tool_calls[0].name in trace_record.plan.allowed_tools,
        "no_empty_final_answer": len(trace_record.output.strip()) > 0,
        "cited_a_source": "source:" in trace_record.output.lower() if trace_record.requires_citation else True,
        "within_step_budget": len(trace_record.steps) <= trace_record.step_budget,
    }
    failed = [name for name, passed in checks.items() if not passed]
    if failed:
        log_eval_failure(trace_record.run_id, failed)
    return checks

These structural checks are cheap and catch a surprising number of real problems: an agent that skipped a required verification step, one that answered without citing a source when it was supposed to, one that blew through its step budget. Layer a smaller number of LLM-as-judge checks on top for semantic correctness (did the answer actually address the question), but keep those on a smaller sample since they cost more and add latency to your eval pipeline, not the live request path.

Run the judge asynchronously, after the response has already gone to the user. You are not gating production traffic on an eval call; you are building a quality signal you can alert on.

Alerting: what should actually page someone

Resist the urge to alert on every metric you just added. Page a human for things that need immediate action, and log everything else to a dashboard you review daily or weekly.

Reasonable paging tier:

• Tool call error rate for a critical tool exceeds a threshold (say, 10%) over a 5-minute window.
• Step count per run doubles versus the trailing 24-hour baseline, which usually means looping.
• Eval failure rate on sampled traffic crosses a threshold, which usually means a prompt or model regression.
• Cost per completed task jumps sharply, which can mean a runaway loop or a pricing/model change you didn't account for.

Reasonable daily-review tier (dashboard, no page):

• Slow drift in output length or tone.
• Gradual increase in escalation/fallback rate.
• Token cost trends over a week.

A useful practice: whenever you ship a prompt change or swap a model version, treat it like a deploy. Tag traces with a prompt version and model version attribute, and watch the metrics above for the new version specifically, side by side with the previous version, for at least a few hours before fully rolling it out. This is the same canary-deploy instinct you'd apply to any backend change, just applied to prompts instead of code.

Logging: what to keep and what to redact

Store full traces, but be deliberate about what's in them. At minimum, log per run: input, final output, every tool call with arguments and results, step count, latency, token counts, cost, and the prompt/model version. Redact PII from logs before they're persisted, not after: strip emails, phone numbers, and any customer identifiers you don't need for debugging, and replace them with a hashed reference you can look up separately if you truly need to reproduce an issue.

Set a retention window that matches your debugging needs, not indefinite retention by default. Two to four weeks of full traces is usually enough to catch and diagnose a regression; older than that, keep the aggregated metrics and drop the raw trace bodies unless you have a specific compliance reason to keep them longer.

Putting it together

A production-ready agent monitoring production setup looks like this in practice:

1. Every agent run emits an OTel trace, with nested spans for planning, each tool call, and each model call.
2. Metrics are extracted from traces in near-real-time: tool error rate per tool, step count per run, cost per completed task, escalation rate.
3. A sample of live traffic runs through structural and semantic evals asynchronously, and eval failures get logged and counted.
4. A small, deliberate set of thresholds pages an on-call human; everything else lands on a dashboard reviewed daily.
5. Prompt and model changes are tagged and compared side by side against the previous version before a full rollout.

None of this is exotic infrastructure. It's the same discipline you'd apply to any production service: trace it, measure it, sample-check the quality, and alert on the signals that actually predict a bad customer experience. The difference with agents is just that "quality" now lives inside the response body, not just the status code, so your instrumentation has to look inside the run, not just at whether it returned.

FAQ

What's the minimum viable monitoring setup for a small team running one agent in production? Start with structured logging of every run: input, output, tool calls, latency, and token cost, even before you add full OTel tracing. Add a simple dashboard for tool error rate and step count per run. That alone catches most early incidents. Add distributed tracing and async evals once you have more than one agent or more than a handful of tools.

Do I need a dedicated LLM observability platform, or can I use my existing APM tool? You can extend an existing APM tool (Datadog, Grafana, Honeycomb) with custom spans and attributes for agent-specific data, which works fine if your team already lives in that tool. Dedicated LLM observability tools add prompt/response viewers, cost breakdowns by model, and eval tooling out of the box, which saves build time if you're starting from scratch. Either path works; consistency with what your on-call team already checks matters more than the specific vendor.

How do I monitor an agent that calls another agent (multi-agent systems)? Treat each sub-agent call as a nested span under the parent run's trace, tagged with which agent handled it. Track the same metrics per sub-agent as you would per tool: error rate, step count, cost. Multi-agent systems fail most often at the handoff boundary, so also track how often a parent agent has to retry or correct a sub-agent's output.

How much sampling is enough for async evals without blowing up cost? Two to five percent of traffic for LLM-as-judge semantic checks is a reasonable starting point for most volumes; scale down further if you're at very high request volume, or run 100% for structural checks since those are cheap. Increase the sample rate temporarily after any prompt or model change to get faster signal on regressions, then drop back to baseline once the new version is stable.

What's the biggest mistake teams make when monitoring agents in production? Only tracking the outer request (latency, status code, uptime) and assuming a 200 response means the agent worked. The most damaging failures are the ones that look completely healthy at the request layer: wrong tool calls, hallucinated arguments, fluent but incorrect answers. If you only have budget to instrument one additional layer beyond the request, instrument the tool call layer first, since that's where most real-world agent failures actually originate.