AI Agentsagent tool errorsLLM tool useerror handlingMCP

Tool Error Recovery in AI Agents

Pramod Dutta · Jul 8, 2026 · 15 min read

Agent tool errors happen constantly in production: a search API times out, a database call returns malformed JSON, a file write hits a permissions error, or the model itself calls a tool with the wrong arguments. Most tutorials show a happy-path agent loop where every tool call succeeds, which is exactly why so many agents fall over the first week they run unsupervised. This article covers how to design tool error recovery into an agent from the start: how to classify errors, how to feed them back to the model in a way it can act on, when to retry versus fail fast, and how to build fallback paths so one broken tool does not take down the whole task.

Why agent tool errors are different from normal exceptions

In a regular application, an exception is something your code catches, logs, and handles with a fixed code path. In an agent loop, the "caller" deciding what to do next is the model itself, reading whatever text you hand it back. That changes the problem in three ways.

First, the model only knows what you tell it. If a tool call raises requests.exceptions.Timeout and you swallow it silently or return an empty string, the model has no idea the call failed and may hallucinate a result or repeat the same broken call forever. Second, the model can only act on errors that are legible in natural language or structured text. A raw Python traceback dumped into the context window burns tokens and rarely helps the model figure out what to do differently. Third, recovery in an agent context often means changing strategy, not just retrying: switching to a different tool, asking a clarifying question, or backing off and doing something else useful while a dependency recovers.

Treat agent tool errors as a first-class part of the interface between your code and the model, not an implementation detail you patch over.

A taxonomy of agent tool errors

Before you can recover from an error, you need to know which bucket it falls into, because each bucket calls for a different response.

Transient/infrastructure errors: timeouts, rate limits, 5xx responses, connection resets. These are usually not the model's fault and are worth retrying with backoff.
Invalid arguments: the model called the tool with a malformed or missing parameter, wrong type, or a value outside an allowed range. These need to go back to the model as a correction request, not a blind retry.
Business-logic failures: the call succeeded but the answer is "no results," "insufficient funds," or "record not found." These are not exceptions in the code sense but do need to be surfaced to the model as meaningful signal.
Permission/auth errors: expired tokens, missing scopes, 403s. These usually cannot be fixed by the model retrying, and need either a refreshed credential or human escalation.
Non-recoverable/fatal errors: a tool that no longer exists, a hard schema mismatch, a downstream service that is fully down. These should short-circuit the loop rather than let the model spin.

Tag every error at the point you catch it with one of these categories. That tag is what drives the retry, backoff, and escalation logic downstream, and it is worth doing even in a quick prototype because it is the difference between an agent that degrades gracefully and one that loops until it burns through its token budget.

Catching errors at the tool boundary

The cleanest place to handle agent tool errors is a thin wrapper around every tool function, not scattered try/except blocks inside business logic. Here is a pattern that works with most agent frameworks and with raw tool-use loops against the Claude API:

import time
import random
from dataclasses import dataclass

@dataclass
class ToolResult:
    ok: bool
    content: str
    error_type: str | None = None
    retryable: bool = False

def call_with_recovery(tool_fn, args, max_retries=3, base_delay=0.5):
    last_error = None
    for attempt in range(max_retries):
        try:
            result = tool_fn(**args)
            return ToolResult(ok=True, content=result)
        except TimeoutError as e:
            last_error = e
            delay = base_delay * (2 ** attempt) + random.uniform(0, 0.25)
            time.sleep(delay)
        except ValueError as e:
            # bad arguments, do not retry blindly, return to model instead
            return ToolResult(
                ok=False,
                content=f"Invalid arguments: {e}",
                error_type="invalid_arguments",
                retryable=False,
            )
        except PermissionError as e:
            return ToolResult(
                ok=False,
                content=f"Permission denied: {e}",
                error_type="auth_error",
                retryable=False,
            )
    return ToolResult(
        ok=False,
        content=f"Tool timed out after {max_retries} attempts: {last_error}",
        error_type="transient",
        retryable=True,
    )

The important part is not the retry loop itself, it is that every exception exits through a ToolResult with a category attached. Whatever calls this function next, whether that is your own orchestration code or the model on the next turn, gets a consistent shape to reason about instead of a bag of exception types.

Feeding errors back to the model

Once you have a categorized error, the next question is what you actually put in the tool result message that goes back into the conversation. Three rules make this work well in practice.

Be specific about what went wrong and what the model can do about it. "Error" is useless. "The search_orders tool requires a customer_id as a string, but received null. Ask the user for their customer ID or order number before retrying" gives the model an actionable next step.

Keep the raw stack trace out of the model's context. Log the full traceback server-side for your own debugging, and put a short, structured summary in the tool result:

{
  "tool": "search_orders",
  "status": "error",
  "error_type": "invalid_arguments",
  "message": "customer_id is required and must be a non-empty string",
  "retryable": false
}

Tell the model explicitly whether retrying makes sense. If you set retryable: false, say so in the text, otherwise the model may try the exact same call again and burn a turn. If you handle retries automatically inside the wrapper (as in the code above), the model should generally never see a raw transient failure at all, it should only see the final outcome after your own retry budget is exhausted.

Retry strategy: when to retry and when to stop

Automatic retries belong in your tool-calling layer, not in the model's reasoning loop, for anything transient. A model deciding "let me try that again" burns a full turn (and the latency and token cost that comes with it) for something a simple backoff loop handles in milliseconds.

A few concrete guidelines:

Use exponential backoff with jitter for rate limits and timeouts, capped at 3 to 5 attempts. Uncapped retries are how agents silently run for twenty minutes on a call that was never going to succeed.
Respect Retry-After headers when a downstream API sends them, rather than guessing your own delay.
Do not retry on 4xx errors other than 429. A 400 or 422 means the request itself was malformed, and retrying the identical request will fail identically. That kind of error should go back to the model as a correction request instead.
Cap total wall-clock time spent on a single tool call, independent of the retry count, so a tool that "succeeds slowly" on attempt five does not blow your latency budget.
Track retry counts per tool per task, not just per call. If the same tool has failed five times across a session, that is a signal to stop retrying and escalate, even if each individual retry budget looks fine in isolation.

def is_retryable(error_type: str) -> bool:
    return error_type in {"transient", "rate_limited"}

def backoff_delay(attempt: int, base: float = 0.5, cap: float = 8.0) -> float:
    return min(cap, base * (2 ** attempt)) + random.uniform(0, 0.25)

Argument validation before the call ever happens

A large share of what looks like "tool errors" is actually the model calling a tool with arguments that violate the schema before the call even reaches the downstream service. Catching this before the network hop saves both latency and a wasted turn.

If you are using JSON Schema for tool definitions (the standard for Claude's tool use and for MCP servers), validate the model's arguments against that schema locally before dispatch:

from jsonschema import validate, ValidationError

def validate_tool_call(schema: dict, arguments: dict) -> str | None:
    try:
        validate(instance=arguments, schema=schema)
        return None
    except ValidationError as e:
        return f"Argument validation failed: {e.message} at {list(e.path)}"

If validation fails, return the error immediately without making the downstream call, and phrase the message around the specific field and constraint that failed, not a generic "bad request." This is also where tightening your tool schema pays off: adding "minLength": 1 to a required string field, or an enum to a status parameter, catches whole classes of malformed calls before they ever hit your tool code.

Fallback tools and degraded paths

Some errors are not retryable and not fixable by better arguments, the tool itself is unavailable. A resilient agent has a fallback path for its most important tools rather than a single point of failure.

Patterns worth building in:

Alternate provider fallback: if your primary web search tool is down, fall back to a secondary search tool with a note in the result that the source changed, so the model can adjust its confidence.
Cached/stale data fallback: if a live lookup fails, return the last known good value from a cache with an explicit "this data may be stale as of <timestamp>" flag, rather than failing the whole task.
Degrade to a narrower capability: if a tool that writes to a database is down, let the agent still read and summarize, and explicitly tell the user that the write step could not complete instead of pretending it did.
Circuit breakers per tool: if a tool has failed N times in a rolling window, stop calling it for a cooldown period and route to fallback logic immediately, rather than paying the timeout cost on every subsequent attempt.

class CircuitBreaker:
    def __init__(self, failure_threshold=5, cooldown_seconds=60):
        self.failure_threshold = failure_threshold
        self.cooldown_seconds = cooldown_seconds
        self.failures = 0
        self.opened_at = None

    def record_failure(self):
        self.failures += 1
        if self.failures >= self.failure_threshold:
            self.opened_at = time.time()

    def record_success(self):
        self.failures = 0
        self.opened_at = None

    def is_open(self) -> bool:
        if self.opened_at is None:
            return False
        if time.time() - self.opened_at > self.cooldown_seconds:
            self.opened_at = None
            self.failures = 0
            return False
        return True

Check is_open() before dispatching a call, and route straight to the fallback or a "tool temporarily unavailable" message if it is tripped. This is the single change that stops a flaky downstream dependency from turning into twenty consecutive slow failures in one agent run.

Idempotency: the error you don't see is worse than the one you do

Timeouts are the most dangerous class of agent tool errors because you often cannot tell whether the underlying action happened. A payment call, an email send, or a database write that times out on the client side may have already succeeded on the server. Retrying blindly can double-charge a customer or send a duplicate email.

Two defenses matter here. First, make write-side tools idempotent wherever the downstream system supports it, using an idempotency key generated once per logical operation and reused across retries:

import uuid

def create_order(customer_id: str, items: list, idempotency_key: str | None = None):
    key = idempotency_key or str(uuid.uuid4())
    # pass `key` through to the downstream API's Idempotency-Key header
    ...

Generate the key once when the model first decides to call the tool, and reuse the same key across your internal retries, not a fresh one per attempt. Second, for tools where idempotency is not available, prefer a read-then-decide pattern: after a timeout, check whether the action already happened (query the order, check the sent-mail log) before deciding to retry, rather than firing the write again blind.

Structured logging and observability for agent tool errors

You cannot fix what you cannot see, and agent failures are notoriously hard to reproduce because the model's next move depends on exactly what text it read. Log every tool call with, at minimum: the tool name, arguments (redact secrets), latency, outcome, error category if any, and the retry count. Correlate all of this with a single trace ID per agent run so you can reconstruct the full sequence of calls and model turns after the fact.

import logging
import json

logger = logging.getLogger("agent.tools")

def log_tool_call(trace_id, tool_name, args, result: ToolResult, latency_ms, attempt):
    logger.info(json.dumps({
        "trace_id": trace_id,
        "tool": tool_name,
        "attempt": attempt,
        "latency_ms": latency_ms,
        "ok": result.ok,
        "error_type": result.error_type,
    }))

Once this is in place, track two metrics over time per tool: error rate and retry rate. A tool with a rising retry rate but stable final error rate is quietly getting slower and costing you latency and token budget even though it "still works." That is usually the first sign a downstream dependency needs attention before it becomes a hard outage.

Escalating to a human

Not every agent tool error should be solved by the agent. Build an explicit escalation path for the categories that a model genuinely cannot fix on its own: expired credentials, a tool that requires an approval the agent does not have, or a task where the agent has exhausted its retry and fallback budget without success.

The failure mode to avoid is the agent pretending it succeeded, or looping silently, when a human needed to be told. When escalation triggers, stop the loop, produce a clear summary of what was attempted and what failed, and hand that back through whatever channel your application uses (a support queue, a Slack message, a status field on the task) rather than leaving the user staring at a stalled process.

Testing your error recovery paths

Error handling code that is never exercised in testing is code you do not actually have. Build a small harness that injects failures into your tool layer deliberately: force a timeout, force a malformed response, force a 429, and confirm the agent does the right thing in each case, not just that it does not crash.

def flaky_tool(fail_times=2):
    calls = {"count": 0}
    def wrapped(**kwargs):
        calls["count"] += 1
        if calls["count"] <= fail_times:
            raise TimeoutError("simulated timeout")
        return "success"
    return wrapped

def test_retry_recovers_after_transient_failures():
    tool = flaky_tool(fail_times=2)
    result = call_with_recovery(tool, {}, max_retries=3)
    assert result.ok

Run this kind of test against every tool wrapper you ship, and add cases for the non-retryable branches too: confirm that a ValueError short-circuits without eating your retry budget, and that a permission error routes to escalation rather than silent retry. This is the same discipline as testing exception handling in any backend service, it is just easy to skip on agent code because the "happy path" demo works the first time you run it.

FAQ

What is the difference between a tool error and a hallucination? A tool error is a real failure signal coming back from the code the agent called: a timeout, a validation failure, a permission error. A hallucination is the model inventing a plausible-looking result when it has no real data to work with, often because a failure was hidden from it rather than surfaced. Good error recovery reduces hallucinations indirectly, because a model that clearly sees "this call failed" is far less likely to fabricate a result than one that receives an empty or ambiguous response.

Should retries happen inside the tool code or inside the agent loop? Transient, infrastructure-level failures should be retried inside the tool wrapper, invisible to the model, because that is cheaper and faster than spending a model turn on it. Failures that require a different strategy, different arguments, or a different tool should surface to the model as a clear error message so it can decide the next step, rather than being retried blindly by your code.

How many retries is reasonable for an agent tool call? Three to five attempts with exponential backoff and jitter is a reasonable default for transient errors, capped by both attempt count and total wall-clock time. Beyond that, the failure is either not actually transient or the downstream system needs longer than your task can afford to wait, and it is better to fail over to a fallback or escalate than to keep retrying.

What should the model see when a tool call fails permanently? A short, structured message stating what failed, the category of failure, and whether retrying makes sense, without the raw stack trace. For example: "The send_email tool failed with a permission error: the configured account does not have send access. This will not succeed on retry; escalate to a human or use an alternate notification method."

Does using MCP change how tool error recovery works? Not fundamentally. MCP standardizes how tools are described and invoked across servers, but the failure modes are the same: timeouts, invalid arguments, permission errors, and business-logic failures still happen at the boundary between your agent and the MCP server. The practices in this article, categorizing errors, validating arguments against the tool's schema before dispatch, retrying only what is safe to retry, and giving the model a clear structured error, apply the same way whether the tool is a local function, a REST API, or an MCP server.

How do I know if my agent's error handling is actually working? Track error rate, retry rate, and escalation rate per tool over time, not just whether the agent "seems to work" in manual testing. If retry rates climb while final error rates stay flat, a dependency is degrading quietly. If escalation rate is zero, either your error handling is genuinely excellent or, more likely, failures are being silently absorbed somewhere instead of surfaced, which is worth auditing directly.