LLM Eval

Evaluating Multi-Agent Systems: Attributing Failure to the Right Agent

Pramod Dutta · Jun 7, 2026 · 14 min read

Why "It Just Doesn't Work" Is Not an Evaluation

You shipped a multi-agent system. A planner agent breaks down the user's request, a retriever agent pulls context, a coder agent writes the fix, and a reviewer agent checks the output before it goes back to the user. It worked beautifully in your demo. Three weeks later, it's wrong 15% of the time, and you have no idea why.

This is the moment most teams get stuck. Someone opens a trace, stares at four agents' worth of tool calls and intermediate reasoning, shrugs, and says "the model's confused." That's not a diagnosis — it's a surrender. Somewhere in that trace, one specific agent made one specific decision that sent the whole pipeline off the rails, and everything downstream just inherited the damage.

Attributing failure to the right agent is the single hardest and most valuable skill in evaluating multi-agent systems. Single-agent evals are comparatively easy: one input, one output, one judge call. Multi-agent evals have to deal with cascading errors, compounding context loss, and agents that each did their individual job "correctly" while the system as a whole still failed. If you can't tell which agent broke first, you can't fix the system — you can only re-roll the dice and hope.

This piece walks through a concrete methodology for isolating failure in multi-agent pipelines: how to instrument agents so failures are traceable, how to build per-agent evaluators instead of one end-to-end judge, how to distinguish an agent's own mistake from a mistake it merely propagated, and how to build a regression suite that catches this stuff before your users do.

Why Multi-Agent Failure Attribution Is Genuinely Different

In a single LLM call, the failure surface is small: bad prompt, bad retrieval, bad output parsing, or the model just got it wrong. You can bisect that in minutes.

In a multi-agent system, failures compound across a chain of decisions, and three properties make attribution much harder than in the single-agent case.

Error propagation. If the planner agent misreads the user's intent, every downstream agent will faithfully execute the wrong plan — and execute it well. The coder agent didn't fail; it perfectly implemented a broken spec. If you only evaluate the final output, you'll blame the coder.
Compounding context loss. Each agent handoff is a compression step. The retriever might summarize ten documents into three bullet points; the planner then reasons only over those three bullets. Information that mattered can silently disappear at any hop, and nothing in the final output tells you where.
Silent partial success. An agent can be 90% right and still poison the outcome. A reviewer agent that catches nine bugs but wrongly approves a tenth, security-relevant bug hasn't "mostly succeeded" — it has failed at the one job that mattered.

Because of this, end-to-end grading of multi-agent systems — just checking whether the final output is good — tells you *that* something broke but almost never tells you *where*. And "where" is the only actionable information you can build on.

Instrumenting the Pipeline: You Can't Attribute What You Can't See

Before you can build any evaluation, you need every agent's inputs and outputs to be individually inspectable. This sounds obvious, but the majority of half-built multi-agent systems only log the final answer, maybe with a debug flag that dumps raw text to stdout. That's not enough.

At minimum, log the following for every agent invocation in the pipeline, as a structured record:

from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any

@dataclass
class AgentTrace:
    run_id: str
    agent_name: str
    step_index: int
    input_payload: dict[str, Any]     # exactly what this agent received
    output_payload: dict[str, Any]    # exactly what this agent produced
    tool_calls: list[dict[str, Any]] = field(default_factory=list)
    tokens_in: int = 0
    tokens_out: int = 0
    latency_ms: float = 0.0
    timestamp: str = field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )

class PipelineTracer:
    def __init__(self, run_id: str):
        self.run_id = run_id
        self.traces: list[AgentTrace] = []

    def record(self, agent_name: str, step_index: int,
               input_payload: dict, output_payload: dict,
               tool_calls: list[dict] | None = None,
               tokens_in: int = 0, tokens_out: int = 0,
               latency_ms: float = 0.0) -> None:
        self.traces.append(AgentTrace(
            run_id=self.run_id,
            agent_name=agent_name,
            step_index=step_index,
            input_payload=input_payload,
            output_payload=output_payload,
            tool_calls=tool_calls or [],
            tokens_in=tokens_in,
            tokens_out=tokens_out,
            latency_ms=latency_ms,
        ))

    def as_dicts(self) -> list[dict]:
        return [t.__dict__ for t in self.traces]

The input_payload field is the one people skip, and it's the most important one. If you only log outputs, you can never tell whether an agent produced a bad answer because it reasoned poorly, or because it was handed bad or incomplete input by the previous agent. Logging both sides of every hop turns your trace into a chain you can walk link by link, checking at each link: given exactly what this agent received, was its output reasonable?

That question — "given exactly what this agent received, was its output reasonable?" — is the entire foundation of failure attribution. It reframes evaluation from "is the final answer good" into a per-agent, per-hop, locally scoped judgment. Once you can ask that question at every hop, attribution becomes a search problem instead of a guessing game.

Build Per-Agent Evaluators, Not One Global Judge

The natural instinct is to write one evaluation prompt that looks at the whole trace and grades the final output. Resist this. A single global judge conflates every agent's contribution and will systematically misattribute blame to whichever agent produced the most visible mistake, even if that agent was just the last one to touch already-corrupted data.

Instead, write one evaluator per agent role, each scoped to that agent's specific job and given only that agent's input/output pair (plus whatever ground truth is relevant to that role). Here's a skeleton that treats each agent as its own evaluation target:

from dataclasses import dataclass

@dataclass
class AgentVerdict:
    agent_name: str
    step_index: int
    passed: bool
    reasoning: str
    failure_category: str | None = None  # e.g. "misread_intent", "bad_tool_call"

PLANNER_RUBRIC = """
You are grading a PLANNER agent in a multi-agent coding assistant.
The planner's only job is to convert a user request into a correct,
complete, ordered list of subtasks for downstream agents.

Given ONLY the user request and the planner's output plan (not what
happened afterward), answer:
1. Does the plan address every explicit requirement in the request?
2. Does the plan omit any implicit requirement a competent engineer
   would infer (e.g. "add tests" when the request is "fix this bug")?
3. Is the ordering of subtasks logically sound?

Do not judge the plan by what later agents did with it. Judge only
whether the plan itself was a correct decomposition of the request.
Return PASS or FAIL, plus a one-sentence reason and a failure
category from: [missing_requirement, wrong_order, scope_creep, none].
"""

def evaluate_planner(user_request: str, plan_output: str, judge_call) -> AgentVerdict:
    prompt = f"{PLANNER_RUBRIC}\n\nUSER REQUEST:\n{user_request}\n\nPLAN:\n{plan_output}"
    result = judge_call(prompt)  # your LLM-as-judge wrapper
    return AgentVerdict(
        agent_name="planner",
        step_index=0,
        passed=result["verdict"] == "PASS",
        reasoning=result["reason"],
        failure_category=result.get("category"),
    )

The critical design choice is in the rubric text itself: "Do not judge the plan by what later agents did with it." Judges naturally want to look ahead at the final outcome and reason backward, which reintroduces exactly the conflation you're trying to eliminate. You have to explicitly instruct the judge to evaluate each agent in isolation, using only the information that agent actually had access to at the time.

Do this for every agent role — retriever, planner, coder, reviewer, whatever your pipeline has — and you get a vector of pass/fail verdicts per run instead of a single number. That vector is what makes attribution possible: when a run fails, you scan the vector and find the first FAIL. Everything after it is suspect but not necessarily broken; everything before it is exonerated.

The First-Failure Rule: Finding Patient Zero

Once you have per-agent verdicts, apply a simple rule: walk the pipeline in execution order and flag the first agent that fails its own rubric. That agent is your primary suspect. Every agent downstream of it inherited a corrupted or incomplete input, so their failures (if any) are likely secondary — they may have made the situation worse, but they didn't start the fire.

def attribute_failure(verdicts: list[AgentVerdict]) -> dict:
    ordered = sorted(verdicts, key=lambda v: v.step_index)
    first_failure = next((v for v in ordered if not v.passed), None)

    if first_failure is None:
        return {"root_cause": None, "status": "all_agents_passed"}

    downstream_failures = [
        v.agent_name for v in ordered
        if v.step_index > first_failure.step_index and not v.passed
    ]

    return {
        "root_cause": first_failure.agent_name,
        "root_cause_category": first_failure.failure_category,
        "root_cause_reasoning": first_failure.reasoning,
        "downstream_agents_also_failed": downstream_failures,
        "status": "attributed",
    }

This is deliberately a simple, mechanical rule rather than another LLM call, and that's the point — you want root-cause attribution to be deterministic and auditable, not another probabilistic judgment layered on top of judgments. The nuance goes into building good per-agent rubrics; the attribution logic on top of them should be boring.

One caveat worth calling out explicitly: "all agents passed" but the final output is still bad is itself a signal. It usually means either your rubrics are too lenient, or the failure is emergent — it lives in the *handoff* between two agents rather than in either agent individually (for example, the planner's step 3 assumes information the retriever agent never surfaced, even though both agents behaved reasonably given their own inputs). When that happens, add a rubric that specifically evaluates handoff compatibility: does agent N's output actually contain what agent N+1 needs.

Distinguishing "Made a Mistake" From "Inherited a Mistake"

There's a subtlety in the first-failure rule that's worth dwelling on, because it's where naive attribution schemes go wrong. Consider a reviewer agent that approves a subtly broken function. Two very different things could have happened:

The coder agent produced correct code implementing a flawed plan, and the reviewer agent correctly verified that the code matched the (flawed) plan. Root cause: the planner.
The coder agent produced code that didn't even match a perfectly good plan, and the reviewer agent failed to catch the deviation. Root cause: the coder, with the reviewer as a secondary failure (it had one job — catch exactly this — and missed it).

Your per-agent rubric for the reviewer needs to ask the right question to distinguish these: not "is the final code good" but "does the code correctly implement the plan it was given, and would a careful reviewer have caught it if not." That's a locally scoped, answerable question. "Is the final code good" is not locally scoped — it smuggles in the planner's mistake and blames the reviewer for not being psychic.

This is why writing rubrics for multi-agent evals is harder than writing rubrics for single-agent evals. You're not just describing what "good" looks like; you're describing what "good, given exactly this input and no more" looks like, for every single role, and you have to keep re-deriving that boundary as your pipeline changes.

A Worked Failure Trace

Concretely, imagine a customer-support pipeline: an intent classifier agent tags the ticket, a knowledge-retriever agent pulls relevant docs, a drafter agent writes a reply, and a compliance-checker agent vets the reply before sending. A user reports: "Your bot told me I could get a refund after 90 days, but your policy is 30 days."

Walking the trace:

Intent classifier tagged the ticket "refund_request" — correct, PASS.
Knowledge-retriever pulled two documents: an old FAQ page (mentions a discontinued 90-day pilot program) and the current refund policy (30 days). It surfaced both, ranked the outdated one first. Rubric asks "did retrieval surface the correct authoritative policy prominently" — this is a FAIL, because ranking is part of its job, not just recall.
Drafter agent read the top-ranked (wrong) document and wrote "90 days" faithfully. Rubric asks "given the input it received, did the drafter accurately reflect it" — PASS, because it accurately used what it was given.
Compliance-checker agent is supposed to cross-reference numeric policy claims against a canonical source, not just the retrieved snippet. It didn't do that cross-check and rubber-stamped the draft. That is also a FAIL — a secondary one, but not a trivial one, because catching exactly this class of error is its entire reason for existing.

First-failure rule points to the retriever as root cause, with the compliance-checker flagged as a downstream secondary failure. That's an actionable finding: fix the retriever's ranking (deprioritize or expire stale documents), and separately harden the compliance-checker so it doesn't depend on the drafter having used the right source. Compare that to an end-to-end judge, which would most likely have said "the bot gave wrong refund information" and pointed everyone at the drafter — the one agent that actually did its job correctly.

Building the Regression Suite

Per-run attribution is diagnostic, not preventive. To stop regressions, turn recurring root causes into a permanent eval suite that runs on every prompt change, every model swap, and every pipeline restructuring.

Seed cases from real failures. Every time you do a root-cause attribution and find a genuine bug, freeze that exact input as a regression case with the expected per-agent verdicts. This is your highest-signal test data — it's not synthetic, it already broke something once.
Weight by root-cause frequency, not by symptom frequency. If ten different final-output failures all trace back to the same retriever ranking bug, that's one bug, not ten. Track unique root causes so you don't over-invest in fixing whichever agent happens to be most visible.
Test agents in isolation and in the full pipeline. Isolated tests (mocked inputs, direct rubric grading) run fast and catch regressions cheaply. Full-pipeline tests catch handoff-level failures that isolated tests structurally cannot see. You need both; neither substitutes for the other.
Re-run attribution whenever you swap models. A cheaper or newer model in one agent slot can shift the failure distribution across the whole pipeline even if that agent's own accuracy improved — because its *outputs* now look different to the next agent in the chain, which may have been implicitly tuned to the old agent's quirks.

class RegressionSuite:
    def __init__(self):
        self.cases: list[dict] = []

    def add_case(self, name: str, inputs: dict, expected_root_cause: str | None,
                 expected_category: str | None = None):
        self.cases.append({
            "name": name,
            "inputs": inputs,
            "expected_root_cause": expected_root_cause,
            "expected_category": expected_category,
        })

    def run(self, pipeline_fn, evaluator_fns: dict) -> list[dict]:
        results = []
        for case in self.cases:
            trace = pipeline_fn(case["inputs"])
            verdicts = [
                evaluator_fns[t.agent_name](t) for t in trace
            ]
            attribution = attribute_failure(verdicts)
            regressed = attribution["root_cause"] != case["expected_root_cause"]
            results.append({
                "case": case["name"],
                "regressed": regressed,
                "attribution": attribution,
            })
        return results

Run this suite as a gate in CI. A pipeline change that fixes the retriever but silently breaks the compliance-checker's cross-referencing should fail the build, not surface three weeks later as a customer complaint.

Common Pitfalls Teams Hit

Grading the whole trace with one judge prompt. This is the single most common mistake and the one this whole article is arguing against. One judge, one holistic score, zero attribution power.
Only logging final outputs. Without intermediate input/output pairs, you can retroactively guess at root cause but never verify it — you're back to eyeballing traces.
Letting the judge see downstream outcomes when grading an upstream agent. If your planner-evaluator prompt includes "and then the coder wrote broken code," the judge will anchor on that and blame the planner even when the plan was fine. Strip everything after the agent's own output from what the judge sees.
Treating "downstream agent also failed" as innocent. A reviewer that let a bug through is still a bug in the reviewer, even if it wasn't the root cause. Track secondary failures separately — they often reveal that your safety nets don't actually catch what they're supposed to catch.
No stable agent identity across pipeline versions. If you refactor and rename "drafter" to "responder," your historical regression cases silently stop matching. Version your agent role names and evaluators together.

Closing: Attribution Is What Makes Evaluation Actionable

The goal of evaluating a multi-agent system was never to produce a single pass/fail score — it was always to produce a decision about what to fix next. A holistic score can tell you the system is at 82% accuracy this week versus 79% last week, but it can't tell you which of your four agents to spend your next sprint on. Per-agent instrumentation, locally scoped rubrics, and a mechanical first-failure rule turn "something is wrong somewhere" into "the retriever's ranking logic is wrong, specifically for stale-document deprioritization" — a finding you can actually act on.

None of this works without a reliable grader sitting underneath every per-agent rubric, which is why LLM-as-a-Judge is the load-bearing technique for the entire methodology described here. Every verdict in this pipeline — whether the planner's decomposition was complete, whether the retriever ranked the right document first, whether the reviewer caught what it should have caught — ultimately comes down to an LLM grading another agent's work against a tightly scoped rubric. Get the judge prompts sloppy or let them see too much downstream context, and your attribution collapses back into guesswork. Get them precise and properly isolated, and failure attribution stops being an art and starts being an engineering discipline you can put a number on.