Monitoring MCP Servers in Production

MCP server monitoring is the practice of tracking tool call latency, error rates, connection health, and resource usage for the Model Context Protocol servers your agents depend on. Once an MCP server moves from a laptop demo to something a production agent calls dozens of times a minute, "it works on my machine" stops being good enough. You need logs you can search, metrics you can graph, and alerts that fire before a customer notices a broken tool. This guide walks through the concrete setup: what to measure, how to instrument a server with structured logs and OpenTelemetry, and how to wire up dashboards and alerts using tools you likely already run.

Why MCP Server Monitoring Matters

An MCP server sits in a strange spot in your stack. It is not quite a normal API (calls come from an LLM's tool-use loop, not a human clicking a button), and it is not quite a normal background worker either (it responds to synchronous requests with a strict timeout budget). That combination creates failure modes that generic APM setups often miss:

• A tool call that hangs stalls the entire agent turn, not just one request.
• A malformed tool response can silently corrupt the model's context for the rest of the conversation.
• Rate limits on an upstream API (a CRM, a search index, a database) get hit in bursts because agents retry aggressively.
• Schema drift between the tool definition and the actual handler produces errors that look like model mistakes but are actually server bugs.

Without monitoring, all of these show up as "the agent is being dumb" complaints, and you end up debugging the wrong layer. With monitoring, you can point at a graph and say "the search_orders tool started timing out at 14:02 when the upstream database connection pool exhausted."

Key Metrics to Track

Before writing any code, decide what "healthy" looks like. For an MCP server, track these at minimum:

• Tool call latency (p50, p95, p99) broken down by tool name, not just server-wide.
• Error rate per tool, split into client errors (bad arguments) and server errors (upstream failures, bugs).
• Timeout rate, tracked separately from generic errors because timeouts usually point at a different root cause (slow upstream, no connection pooling).
• Connection/session count, especially if your server is stateful (SSE or WebSocket transport) rather than stateless stdio.
• Tool call volume, which tells you which tools are actually used and which ones you can deprecate.
• Payload size, in and out. Oversized tool responses are a common, sneaky cause of slow agent turns and blown context windows.
• Upstream dependency health, if your tools wrap a database, a REST API, or another service.

Resist the urge to track everything from day one. Start with latency, error rate, and volume per tool. Add the rest once you have a baseline.

Structured Logging for Tool Calls

The single highest-leverage change you can make to an MCP server is switching from print-style logging to structured logs with a consistent request ID. Every tool call should log at least: the tool name, arguments (redacted where needed), duration, outcome, and a correlation ID you can also see in traces.

Here is a Python example using the structlog library wrapped around a generic MCP tool handler pattern:

import time
import uuid
import structlog

logger = structlog.get_logger()

def instrumented_tool(tool_name):
    def decorator(fn):
        def wrapper(*args, **kwargs):
            call_id = str(uuid.uuid4())
            start = time.monotonic()
            log = logger.bind(
                call_id=call_id,
                tool_name=tool_name,
                args_keys=list(kwargs.keys()),
            )
            log.info("tool_call_started")
            try:
                result = fn(*args, **kwargs)
                duration_ms = (time.monotonic() - start) * 1000
                log.info(
                    "tool_call_completed",
                    duration_ms=round(duration_ms, 2),
                    outcome="success",
                )
                return result
            except TimeoutError:
                duration_ms = (time.monotonic() - start) * 1000
                log.warning(
                    "tool_call_completed",
                    duration_ms=round(duration_ms, 2),
                    outcome="timeout",
                )
                raise
            except Exception as exc:
                duration_ms = (time.monotonic() - start) * 1000
                log.error(
                    "tool_call_completed",
                    duration_ms=round(duration_ms, 2),
                    outcome="error",
                    error_type=type(exc).__name__,
                )
                raise
        return wrapper
    return decorator


@instrumented_tool("search_orders")
def search_orders(customer_id: str, status: str = "any"):
    # actual tool logic here
    return query_orders_db(customer_id, status)

Two things matter here. First, call_id lets you grep one tool invocation across every log line it produced, even if the handler calls other functions that log independently. Second, the outcome is always logged, even on the happy path, so your log volume tells you the true call rate, not just the error rate.

If your MCP server is written in TypeScript, the same pattern applies with pino instead of structlog:

import pino from "pino";
import { randomUUID } from "crypto";

const logger = pino();

function instrumentedTool(toolName: string, fn: (args: any) => Promise<any>) {
  return async (args: any) => {
    const callId = randomUUID();
    const start = Date.now();
    const log = logger.child({ callId, toolName });
    log.info({ argsKeys: Object.keys(args) }, "tool_call_started");
    try {
      const result = await fn(args);
      log.info({ durationMs: Date.now() - start, outcome: "success" }, "tool_call_completed");
      return result;
    } catch (err: any) {
      log.error(
        { durationMs: Date.now() - start, outcome: "error", errorType: err?.name },
        "tool_call_completed"
      );
      throw err;
    }
  };
}

Ship these logs somewhere searchable. A local JSON file works for a demo; for production, ship to whatever log aggregator your team already runs (Loki, Datadog, CloudWatch Logs, or plain Elasticsearch). The point is that when an agent conversation goes wrong, you can pull the exact tool calls that happened during that turn using the call ID that your agent framework or client passed along in its request metadata.

Instrumenting an MCP Server with OpenTelemetry

Logs tell you what happened; traces tell you where the time went. If your MCP server calls a database, an internal microservice, and a third-party API inside a single tool handler, a trace shows you the breakdown instantly instead of you guessing from timestamps in log lines.

A minimal OpenTelemetry setup for a Python MCP server:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="localhost:4317", insecure=True))
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)

tracer = trace.get_tracer("mcp.tools")


def traced_tool(tool_name):
    def decorator(fn):
        def wrapper(*args, **kwargs):
            with tracer.start_as_current_span(f"mcp.tool.{tool_name}") as span:
                span.set_attribute("mcp.tool.name", tool_name)
                span.set_attribute("mcp.tool.arg_count", len(kwargs))
                try:
                    result = fn(*args, **kwargs)
                    span.set_attribute("mcp.tool.outcome", "success")
                    return result
                except Exception as exc:
                    span.set_attribute("mcp.tool.outcome", "error")
                    span.record_exception(exc)
                    raise
        return wrapper
    return decorator

Point the OTLP exporter at whatever collector you already run (an OpenTelemetry Collector, Jaeger, Tempo, or a hosted APM vendor's OTLP endpoint). Once wired up, nest additional spans inside each tool for the database call, the HTTP request to an upstream API, and any parsing or validation step. That nested structure is what turns "the search_orders tool is slow" into "the Postgres query inside search_orders is slow because it is missing an index," without adding a single print statement.

Building a Health Check Endpoint

Most MCP servers running over HTTP or SSE transport should expose a lightweight health endpoint separate from the MCP protocol endpoint itself, so your load balancer or orchestrator can check liveness without going through a full tool-call round trip.

from fastapi import FastAPI, Response
import time

app = FastAPI()
start_time = time.monotonic()


@app.get("/healthz")
async def healthz():
    checks = {
        "database": check_database_connection(),
        "upstream_api": check_upstream_api(),
    }
    healthy = all(checks.values())
    uptime_seconds = time.monotonic() - start_time
    body = {
        "status": "ok" if healthy else "degraded",
        "checks": checks,
        "uptime_seconds": round(uptime_seconds, 1),
    }
    return Response(
        content=str(body),
        status_code=200 if healthy else 503,
        media_type="application/json",
    )

Two health check tiers are worth separating:

1. Liveness: is the process up and able to respond at all. Keep this cheap, no dependency calls.
2. Readiness: are the dependencies (database, upstream APIs, credentials) actually reachable. This is where you run the checks shown above.

Feed readiness into your deployment platform so a bad rollout gets pulled out of rotation automatically instead of serving broken tool calls to every agent that connects.

Alerting on Failure Patterns

Metrics and traces are only useful if someone gets paged before the damage compounds. A few alert rules cover most real incidents on MCP servers:

• Error rate spike per tool: alert if a single tool's error rate crosses a threshold (for example, more than 10% of calls failing over a 5 minute window), rather than alerting on the server's aggregate error rate, which dilutes single-tool problems.
• p99 latency breach: alert when p99 latency for any tool exceeds your agent framework's timeout minus a safety margin. If your client times out tool calls at 30 seconds, alert at 20 seconds so you catch it before users do.
• Timeout rate: alert on timeouts specifically, separate from generic 5xx-style errors, since timeouts usually mean a stuck upstream connection or missing connection pool limits.
• Zero traffic: alert if a tool that normally gets steady volume drops to zero calls, which often means the MCP client lost its connection or the tool got deregistered by accident.
• Upstream dependency down: alert directly on the readiness check results from your health endpoint, not just on symptoms downstream.

A simple Prometheus alerting rule for the error-rate case looks like this:

groups:
  - name: mcp-server-alerts
    rules:
      - alert: MCPToolErrorRateHigh
        expr: |
          sum(rate(mcp_tool_calls_total{outcome="error"}[5m])) by (tool_name)
          /
          sum(rate(mcp_tool_calls_total[5m])) by (tool_name)
          > 0.10
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "MCP tool {{ $labels.tool_name }} error rate above 10%"

That rule assumes you are exporting a mcp_tool_calls_total counter labeled with tool_name and outcome, which brings us to metrics export.

Exporting Metrics with Prometheus

Wrap the same instrumented decorator pattern from earlier with prometheus_client counters and histograms:

from prometheus_client import Counter, Histogram, start_http_server

TOOL_CALLS = Counter(
    "mcp_tool_calls_total",
    "Total MCP tool calls",
    ["tool_name", "outcome"],
)

TOOL_DURATION = Histogram(
    "mcp_tool_call_duration_seconds",
    "MCP tool call duration",
    ["tool_name"],
)


def metered_tool(tool_name):
    def decorator(fn):
        def wrapper(*args, **kwargs):
            with TOOL_DURATION.labels(tool_name=tool_name).time():
                try:
                    result = fn(*args, **kwargs)
                    TOOL_CALLS.labels(tool_name=tool_name, outcome="success").inc()
                    return result
                except Exception:
                    TOOL_CALLS.labels(tool_name=tool_name, outcome="error").inc()
                    raise
        return wrapper
    return decorator


# expose /metrics for Prometheus to scrape
start_http_server(9100)

Stack the decorators (logging, tracing, metrics) on the same tool handler, or fold them into one combined decorator if the stack trace noise bothers you. Once metrics are flowing, a Grafana dashboard with three panels covers 90% of day-to-day monitoring needs: call volume per tool, p50/p95/p99 latency per tool, and error rate per tool, all with a time range selector so you can zoom into an incident window.

Handling Timeouts and Retries Safely

Agents retry failed tool calls more aggressively than typical API clients, because the model sees an error and often just tries again with slightly different arguments. That behavior turns a brief upstream blip into a retry storm if your server does not protect itself.

Two defenses matter most:

• Set an explicit per-tool timeout shorter than your MCP client's overall timeout, so a slow tool fails fast and predictably instead of eating the entire agent turn's time budget.
• Add a circuit breaker around upstream calls so that once an upstream dependency is clearly down, your server returns a fast, clear error instead of letting every retry queue up behind a dead connection.

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, reset_after_seconds=30):
        self.failure_threshold = failure_threshold
        self.reset_after_seconds = reset_after_seconds
        self.failure_count = 0
        self.open_until = 0

    def call(self, fn, *args, **kwargs):
        if time.monotonic() < self.open_until:
            raise RuntimeError("circuit_open: upstream marked unhealthy")
        try:
            result = fn(*args, **kwargs)
            self.failure_count = 0
            return result
        except Exception:
            self.failure_count += 1
            if self.failure_count >= self.failure_threshold:
                self.open_until = time.monotonic() + self.reset_after_seconds
            raise

Log every time the circuit opens and closes. That log line, on its own, is often the fastest way to explain a cluster of agent failures during an incident review: "the circuit breaker opened at 09:14 because the payments API started timing out."

Security Monitoring for MCP Servers

MCP servers frequently hold credentials for the systems they wrap: database connections, API keys, internal service tokens. Monitoring here is as much about audit as about uptime.

• Log every tool call's calling identity (which agent, which user session, which API key) alongside the tool name and arguments, so you can answer "who called this tool and when" during a security review.
• Redact sensitive argument values before they hit your log pipeline. A regex-based redactor for common patterns (emails, tokens, card numbers) run on log fields before they are written is cheap insurance.
• Track authorization failures as their own metric, separate from generic errors, since a spike there can indicate a misconfigured client or a credential that leaked.
• Rate-limit per calling identity, not just globally, so one misbehaving agent instance cannot starve the tool for everyone else.

Putting It Together

A production-ready MCP server does not need a large observability team behind it. It needs, at minimum: structured logs with a correlation ID per tool call, metrics exported per tool (volume, latency, errors), a readiness health check that actually exercises dependencies, and two or three alert rules tuned to your agent client's timeout budget. Add distributed tracing once you have more than one hop inside a tool handler, since that is when "which part was slow" stops being obvious from logs alone.

Start with the logging decorator shown above. It is the cheapest change with the highest immediate payoff, because the first time an agent conversation goes sideways in production, a searchable log of every tool call it made will save you more time than any dashboard.

FAQ

What is the difference between monitoring an MCP server and a regular REST API? The core techniques (structured logs, metrics, tracing, health checks) are the same. The difference is in what breaks: MCP servers are called by an LLM's tool-use loop rather than a human-driven client, so retry storms, oversized responses that blow the model's context, and single stuck tool calls that stall an entire agent turn are more common failure modes than they are for typical web APIs.

Do I need OpenTelemetry for a small MCP server? Not necessarily on day one. Structured logging with a correlation ID and a handful of Prometheus counters covers most needs for a server with one or two tools. Add tracing once a tool handler starts calling multiple downstream services and you need to see the time breakdown between them.

How do I monitor an MCP server that uses stdio transport instead of HTTP? Stdio servers cannot expose an HTTP health endpoint directly, but they can still emit structured logs to stderr or a file, and you can still wrap tool handlers with the same logging and metrics decorators shown above. Ship those logs to your aggregator through whatever process supervisor launches the server, and push metrics through a separate lightweight HTTP server started inside the same process if you need Prometheus scraping.

What is a reasonable timeout for an MCP tool call? There is no universal number since it depends entirely on what the tool does, but the important rule is relative, not absolute: the per-tool timeout inside your server should always be shorter than the timeout your MCP client enforces on the whole call, with enough margin that a clean timeout error can propagate back before the client gives up on its own.

Should every tool call be logged, even successful ones? Yes. Logging only errors means you cannot tell the difference between "this tool is barely used" and "this tool is failing silently before it even gets logged." Logging every call, tagged with its outcome, gives you an accurate baseline for both volume and error rate.

Can I reuse my existing APM tooling for MCP servers? In most cases yes. If your team already runs Datadog, New Relic, Grafana stack, or a similar APM platform, point the same OpenTelemetry exporters and Prometheus scrape configs at your MCP server rather than standing up separate infrastructure. The instrumentation patterns in this guide work with any OTLP-compatible or Prometheus-compatible backend.