DeepEval

DeepEval for Agents: Testing Tool Use and Task Completion

Ira Menon · Jun 13, 2026 · 15 min read

Your agent passed every unit test and still failed in production

Here is a scenario that shows up in almost every team building agents: the LLM call works, the prompt is well-tuned, the demo looks great — and then a user asks the agent to "cancel my subscription and email me a confirmation," and the agent calls the refund tool instead of the cancellation tool, skips the email, and confidently tells the user everything is done.

Traditional software tests do not catch this. You cannot write a simple assert response == expected_output for an agent, because the "correct" output is not a fixed string — it is a sequence of decisions: which tool to call, with what arguments, in what order, and whether the final outcome actually satisfies the user's request. This is a fundamentally different testing problem than evaluating a single LLM response, and it is why generic LLM evaluation metrics like answer relevancy or faithfulness are not enough once you move from "chatbot that answers questions" to "agent that takes actions."

DeepEval, the open-source LLM evaluation framework, has two metrics purpose-built for this: ToolCorrectnessMetric and TaskCompletionMetric. Together they let you test the two things that actually matter for an agent — did it use its tools correctly, and did it finish the task it was given. In this article we will build a working test suite for a simple agent, use both metrics, wire them into pytest, and talk about where each one falls short so you do not over-trust a passing score.

Why agent testing is different from LLM testing

A standard LLM evaluation checks a single input-output pair. You give the model a prompt, it returns text, and you score that text against criteria like relevance, faithfulness to a retrieved context, or grammatical correctness. That is a one-shot judgment.

An agent is a loop. It receives a goal, decides on an action (usually a tool call), observes the result, and repeats until it decides the goal is met. Testing an agent means testing that entire trajectory, not just the final sentence it prints. Two failure modes are unique to agents and deserve their own metrics:

Wrong tool selection. The agent picks search_web when it should have picked query_database, or it calls the right tool with the wrong arguments, or it calls three tools when one would do.
Incomplete task execution. The agent calls all the right tools, gets all the right data back, but never actually completes the user's underlying request — it retrieves the refund policy but forgets to actually issue the refund.

ToolCorrectnessMetric targets the first failure mode. TaskCompletionMetric targets the second. You typically want both running in your test suite, because an agent can score perfectly on tool selection while still failing to complete the task (it called the right tools but stopped too early), and it can complete the task while having taken an inefficient or risky path to get there.

Setting up DeepEval for agent testing

Install DeepEval and set your model provider credentials. DeepEval defaults to using OpenAI models as the judge (the "LLM-as-a-judge" that scores your agent), but it supports custom judge models too.

pip install -U deepeval
export OPENAI_API_KEY="sk-..."

The two building blocks you need are LLMTestCase (for a single, already-completed interaction) and ToolCall (for representing what tool was invoked and with what parameters). Here is the minimal shape:

from deepeval.test_case import LLMTestCase, ToolCall

test_case = LLMTestCase(
    input="Cancel my subscription and email me a confirmation.",
    actual_output="Your subscription has been cancelled and a confirmation email is on its way.",
    tools_called=[
        ToolCall(name="cancel_subscription", input_parameters={"user_id": "u_123"}),
        ToolCall(name="send_email", input_parameters={"template": "cancellation_confirmed"}),
    ],
    expected_tools=[
        ToolCall(name="cancel_subscription"),
        ToolCall(name="send_email"),
    ],
)

Notice that tools_called captures what the agent actually did at runtime, and expected_tools is the ground truth you define as the test author. This is the same input/expected-output split you know from unit testing — you just express it in terms of tool calls instead of return values.

Testing tool selection with ToolCorrectnessMetric

ToolCorrectnessMetric compares tools_called against expected_tools and produces a score based on overlap. Unlike most DeepEval metrics, it does not need an LLM judge by default — it is a deterministic, rule-based comparison, which makes it fast and cheap to run on every commit.

from deepeval import evaluate
from deepeval.metrics import ToolCorrectnessMetric
from deepeval.test_case import LLMTestCase, ToolCall

metric = ToolCorrectnessMetric(threshold=0.7, include_reason=True)

test_case = LLMTestCase(
    input="What if these shoes don't fit?",
    actual_output="We offer a 30-day full refund on all unworn footwear.",
    tools_called=[ToolCall(name="search_policy_db"), ToolCall(name="format_response")],
    expected_tools=[ToolCall(name="search_policy_db")],
)

evaluate(test_cases=[test_case], metrics=[metric])
print(metric.score)
print(metric.reason)

A few constructor arguments matter a lot in practice:

`should_exact_match` (default False) — if True, the agent's tool list must match expected_tools exactly, in the same set, with nothing extra and nothing missing. Use this for agents where calling an unnecessary tool is itself a bug (for example, an agent that should never call a delete_record tool for a read-only query).
`should_consider_ordering` (default False) — if True, the sequence matters, not just the set of tools. This is important for workflows where calling tools out of order breaks the task, like validating a payment method before charging it rather than after.
`evaluation_params` — pass ToolCallParams.INPUT_PARAMETERS if you also want to score whether the agent called the tool with the correct arguments, not just the correct name. This catches a subtler bug: the agent picked the right tool but passed the wrong user_id or a malformed date string.
`available_tools` — an optional list of every tool the agent had access to. This lets the metric reason about whether the agent ignored a better tool it had available, which is useful context for the reason string even though it does not change the core scoring math.

Start with the loose defaults (should_exact_match=False, should_consider_ordering=False) when you first introduce this metric, then tighten them as you learn which of your agent's flows are order-sensitive or zero-tolerance. If you tighten everything on day one, you will get a wall of false failures for tool calls that are perfectly fine just because they happened in a different order.

Testing outcomes with TaskCompletionMetric

Tool correctness tells you whether the agent used its tools well. It does not tell you whether the user's actual goal was met. An agent can call every correct tool and still fail the task — for example, it fetches the refund eligibility, confirms the order is eligible, and then just... stops, without actually processing the refund.

TaskCompletionMetric is an LLM-as-a-judge metric that looks at the full trace of an agent run — the input task, the sequence of steps taken, and the final outcome — and scores how well the outcome satisfies the task. Unlike ToolCorrectnessMetric, this one needs a judge model because "did this actually solve the user's problem" is a judgment call, not a set comparison.

from deepeval.metrics import TaskCompletionMetric
from deepeval.test_case import LLMTestCase

task_completion = TaskCompletionMetric(
    threshold=0.7,
    model="gpt-4o",
    include_reason=True,
)

test_case = LLMTestCase(
    input="Cancel my subscription and email me a confirmation.",
    actual_output="I've looked up your account and confirmed you are eligible to cancel.",
    tools_called=[ToolCall(name="lookup_account"), ToolCall(name="check_eligibility")],
)

task_completion.measure(test_case)
print(task_completion.score)
print(task_completion.reason)

Run that test case and you should see a low score with a reason explaining that the agent verified eligibility but never actually called a cancellation tool or sent the confirmation email — the task was started but not completed. That is exactly the failure mode that a tool-set comparison alone would not catch if your expected_tools list happened to only include the lookup and eligibility steps.

TaskCompletionMetric works best when you give it visibility into the agent's full trace rather than just a single input/output pair, because task completion is inherently about the whole trajectory. DeepEval's tracing integration makes this straightforward with the @observe decorator, which wraps your agent's functions (including individual tool calls) and streams the trace to DeepEval automatically.

from deepeval.tracing import observe, update_current_span
from deepeval.metrics import TaskCompletionMetric

@observe(type="tool", description="Cancel a user's active subscription.")
def cancel_subscription(user_id: str) -> dict:
    # real cancellation logic here
    return {"status": "cancelled", "user_id": user_id}

@observe(type="tool", description="Send a templated confirmation email.")
def send_email(template: str, user_id: str) -> bool:
    # real email-sending logic here
    return True

@observe(metrics=[TaskCompletionMetric(threshold=0.7, model="gpt-4o")])
def subscription_agent(query: str, user_id: str) -> str:
    cancel_subscription(user_id)
    send_email(template="cancellation_confirmed", user_id=user_id)
    result = "Your subscription has been cancelled and a confirmation email is on its way."
    update_current_span(input=query, output=result)
    return result

Because each tool call is its own @observe-wrapped span, DeepEval can see not just the final answer but the shape of the trace that produced it — which tools ran, in what order, and what they returned. That is the signal TaskCompletionMetric needs to reason about whether the task was genuinely completed rather than just narrated.

There's also a subtler class of bug that slips past a metric checking only tool *names*: the agent picks the exactly right tool but hands it garbage. It calls cancel_subscription — correct choice — with the wrong user_id pulled from a stale part of the conversation, or it calls issue_refund with an amount that does not match the order total. If your expected_tools list only carries tool names, ToolCorrectnessMetric will happily give this a perfect score, because as far as the set comparison is concerned, the right tool fired.

This is what the evaluation_params argument is for. Pass ToolCallParams.INPUT_PARAMETERS and the metric starts comparing the actual arguments the agent passed against the arguments you specified on expected_tools, not just the tool name.

from deepeval.metrics import ToolCorrectnessMetric
from deepeval.metrics.tool_correctness.tool_correctness import ToolCallParams
from deepeval.test_case import LLMTestCase, ToolCall

test_case = LLMTestCase(
    input="Refund order #8891, it arrived damaged.",
    actual_output="I've issued a refund for order #8891.",
    tools_called=[
        ToolCall(name="issue_refund", input_parameters={"order_id": "8891", "amount": 41.00}),
    ],
    expected_tools=[
        ToolCall(name="issue_refund", input_parameters={"order_id": "8891", "amount": 58.50}),
    ],
)

metric = ToolCorrectnessMetric(
    threshold=0.9,
    evaluation_params=[ToolCallParams.INPUT_PARAMETERS],
)
metric.measure(test_case)
print(metric.score, metric.reason)

Here the tool name matches perfectly, but the refund amount is wrong — a bug that would otherwise sail through a name-only comparison and only surface later as a support escalation or a finance discrepancy. Argument-level checking costs you a small amount of extra fixture-writing (you now have to know and encode the correct expected arguments, not just the correct tool), but for anything that touches money, PII, or irreversible state changes, it is worth the extra specificity. Reserve it for your highest-risk flows rather than applying it everywhere by default, since writing exact expected arguments for every low-stakes tool call (like a read-only search_faq lookup) adds fixture maintenance for little payoff.

Beyond single-turn tool and argument accuracy, most real agents do not resolve a request in a single exchange. A booking agent might need to ask a clarifying question, wait for the user's answer, and only then call a tool. Testing this properly means testing the conversation as a unit, not just the final turn in isolation — an agent can look perfect on the last message while having completely lost track of something the user said two turns earlier.

DeepEval handles this with ConversationalTestCase, which wraps an ordered list of Turn objects so a metric can reason across the whole exchange instead of a single input/output pair.

from deepeval.test_case import ConversationalTestCase, Turn, ToolCall
from deepeval.metrics import TaskCompletionMetric

turns = [
    Turn(role="user", content="I need to move my dentist appointment."),
    Turn(
        role="assistant",
        content="Sure — I see one booked for July 10th at 2pm. What day works better?",
        tools_called=[ToolCall(name="get_upcoming_appointments", input_parameters={"user_id": "u_88"})],
    ),
    Turn(role="user", content="Move it to the following week, same time."),
    Turn(
        role="assistant",
        content="Done — your appointment is now July 17th at 2pm.",
        tools_called=[
            ToolCall(
                name="reschedule_appointment",
                input_parameters={"appointment_id": "apt_552", "new_date": "2025-07-17", "new_time": "14:00"},
            )
        ],
    ),
]

convo_test_case = ConversationalTestCase(turns=turns)

metric = TaskCompletionMetric(threshold=0.8, model="gpt-4o")
metric.measure(convo_test_case)
print(metric.score, metric.reason)

This is where you catch the failure modes that single-turn testing structurally cannot see: the agent asking a clarifying question and then ignoring the answer, resolving a relative date like "the following week" against the wrong reference point, or dropping a constraint the user mentioned early in the conversation (like "keep it under $50") by the time it actually calls a tool three turns later. If you only ever assemble LLMTestCase objects from the final turn of a conversation, none of that state-tracking behavior is visible to your test suite at all — it is worth building at least a handful of multi-turn ConversationalTestCase fixtures for any agent that holds a conversation longer than one exchange.

Running both metrics together over a dataset

Testing one hand-written example is a sanity check, not a test suite. For real coverage, build a Golden dataset — a set of representative user requests — and run your agent against every one of them with both metrics attached.

from deepeval.dataset import Golden, EvaluationDataset
from deepeval.metrics import ToolCorrectnessMetric, TaskCompletionMetric
from deepeval.test_case import ToolCall

goldens = [
    Golden(
        input="Cancel my subscription and email me a confirmation.",
        expected_tools=[ToolCall(name="cancel_subscription"), ToolCall(name="send_email")],
    ),
    Golden(
        input="What's your return policy on electronics?",
        expected_tools=[ToolCall(name="search_policy_db")],
    ),
    Golden(
        input="Upgrade me to the annual plan and refund the difference.",
        expected_tools=[
            ToolCall(name="change_plan"),
            ToolCall(name="calculate_proration"),
            ToolCall(name="issue_refund"),
        ],
    ),
]

dataset = EvaluationDataset(goldens=goldens)

tool_metric = ToolCorrectnessMetric(threshold=0.7)
task_metric = TaskCompletionMetric(threshold=0.7, model="gpt-4o")

for golden in dataset.evals_iterator(metrics=[tool_metric, task_metric]):
    subscription_agent(golden.input, user_id="u_test_001")

The evals_iterator pattern is worth calling out because it inverts the usual test structure. Instead of you calling metric.measure(test_case) manually for each golden, DeepEval iterates the dataset, runs your instrumented agent function for each input, captures the trace via @observe, and scores it against every metric you passed in. This is what makes it practical to run 50 or 200 goldens as part of CI rather than a handful of examples you eyeball by hand.

Wiring it into pytest

Once you have a dataset and metrics, drop them into a pytest test so agent regressions fail your build the same way a broken unit test would.

import pytest
from deepeval import assert_test
from deepeval.test_case import LLMTestCase, ToolCall
from deepeval.metrics import ToolCorrectnessMetric, TaskCompletionMetric

@pytest.mark.parametrize("query,expected_tools", [
    ("Cancel my subscription and email me a confirmation.",
     [ToolCall(name="cancel_subscription"), ToolCall(name="send_email")]),
    ("What's your return policy on electronics?",
     [ToolCall(name="search_policy_db")]),
])
def test_agent_tool_use_and_completion(query, expected_tools):
    output = subscription_agent(query, user_id="u_test_001")

    test_case = LLMTestCase(
        input=query,
        actual_output=output,
        tools_called=[ToolCall(name=t.name) for t in expected_tools],  # replace with real captured calls
        expected_tools=expected_tools,
    )

    assert_test(test_case, [
        ToolCorrectnessMetric(threshold=0.7),
        TaskCompletionMetric(threshold=0.7, model="gpt-4o"),
    ])

Run it the same way you run any other suite:

deepeval test run test_agent.py

deepeval test run is a thin wrapper around pytest that also gives you DeepEval's test report output — pass/fail per metric, per test case, with the reason string attached. That reason string is the part worth reading even when a test passes, because it tells you why the judge scored it that way, which is invaluable when you are tuning thresholds.

Reading scores correctly and avoiding false confidence

A few practical lessons from putting these metrics into a CI pipeline:

Thresholds are not universal. A threshold=0.7 on TaskCompletionMetric for a simple FAQ agent means something different than the same threshold for a multi-step refund workflow. Start every new metric at a low threshold, look at real scores and reasons across a batch of goldens, and raise the bar once you know what "good" actually looks like for that specific agent.
`ToolCorrectnessMetric` cannot judge intent, only sets and (optionally) order and arguments. If your expected_tools list is wrong or incomplete, the metric will happily reward a broken agent for matching a broken spec. The metric is only as good as the goldens you write, which is exactly the same problem test-driven development has always had — garbage expectations produce garbage confidence.
`TaskCompletionMetric` is a judge, and judges can disagree. Because it uses an LLM to reason about the trace, running it twice on the same trace can occasionally produce slightly different scores or reasons, especially near the threshold boundary. Treat borderline scores (say, 0.65–0.75 against a 0.7 threshold) as a signal to look closer, not as a definitive pass or fail.
Combine both metrics rather than picking one. Tool correctness without task completion tells you the agent followed the recipe but says nothing about whether the dish was edible. Task completion without tool correctness tells you the outcome looked fine but hides the fact that the agent got there through an inefficient, expensive, or risky sequence of tool calls that happened to still work this time. Neither metric alone gives you the full picture; running them side by side does.
Instrument tools with `@observe(type="tool", ...)` from the start. Retrofitting tracing onto an agent you already shipped is more work than adding it while you build. The description argument you pass to @observe also becomes part of the context the judge model sees, so a clear one-line description of what each tool does measurably improves the quality of TaskCompletionMetric reasoning.

Beyond unit tests: making this part of your workflow

None of this replaces watching real users interact with your agent — DeepEval's confidently scores are only as trustworthy as the goldens and thresholds you feed it, and no automated metric fully substitutes for a human reading a transcript occasionally. But once you have even a modest golden set covering your agent's core flows, ToolCorrectnessMetric and TaskCompletionMetric catch the exact class of regression that slips past standard software tests: the agent that still runs without errors, still returns plausible-sounding text, and still quietly does the wrong thing.

The pattern scales the same way traditional test suites do. Start with a handful of hand-written goldens for your riskiest flows — payments, cancellations, anything irreversible — wire assert_test into pytest, run it in CI on every pull request that touches agent logic, and grow the dataset as you discover new edge cases in production. Treat a failing TaskCompletionMetric score the way you would treat a failing integration test: as a signal that something in the trace changed in a way that matters, worth reading the reason field for before you touch the threshold.

If you want a guided, hands-on walkthrough of building this evaluation pipeline end to end — writing goldens, instrumenting real agents with @observe, tuning thresholds, and hooking everything into CI — our DeepEval Tutorial course on teachyou.ai covers exactly this, with working code you can adapt directly to your own agent stack.