DeepEval for Multi-Modal Apps: Testing Vision-Language Outputs

Why Testing Vision-Language Outputs Breaks Your Old Eval Stack

If you've shipped a text-only LLM feature before, you probably have a comfortable eval routine: feed in a prompt, compare the output against an expected answer or a rubric, score it, move on. Then someone on your team ships a feature where the model looks at a screenshot, a product photo, a chart, or a medical scan and describes what it sees, and your entire eval stack quietly stops working.

The problem isn't that vision-language models (VLMs) are harder to call. It's that grounding an evaluation now requires the image itself, not just the text around it. A response like "the chart shows a 12% increase in Q3" is either faithful to the image or it's a hallucination, and no text-only metric can tell the difference without looking at the picture. GPT-4V-style models, Gemini's vision variants, and Claude's multi-modal capabilities all suffer from a version of the same failure mode: they describe what's plausible instead of what's actually there, especially with charts, dense tables, small text in screenshots, or subtle visual details.

DeepEval, the open-source LLM evaluation framework, has added multi-modal support specifically to close this gap. Instead of scoring text against text, you can now score text against images, or image-generation output against a source image, using the same metric-driven workflow you already use for RAG pipelines and chatbots. This article walks through what multi-modal testing in DeepEval actually looks like in practice: the data structures, the metrics that matter, and the pitfalls that show up the moment real images enter your test suite.

The Core Building Block: MLLMTestCase

DeepEval's text-only evaluations revolve around LLMTestCase — an object holding your input, actual output, expected output, and retrieval context as strings. For multi-modal evaluation, DeepEval introduces MLLMTestCase, which extends that idea to accept a list of inputs that can mix text and images.

The key design choice is that both input and actual_output become lists, where each item is either a plain string or an MLLMImage object. This matters because real vision-language interactions are rarely "one image, one question" — they're often interleaved, like "here's a screenshot of a dashboard [image], and here's a follow-up screenshot after I clicked submit [image], did the value update correctly?"

from deepeval.test_case import MLLMTestCase, MLLMImage

test_case = MLLMTestCase(
    input=[
        "Describe what changed between these two dashboard states.",
        MLLMImage(url="./screenshots/before.png", local=True),
        MLLMImage(url="./screenshots/after.png", local=True),
    ],
    actual_output=[
        "The revenue widget updated from $12,400 to $14,900, and a new alert badge appeared on the notifications icon.",
    ],
)

MLLMImage accepts either a local file path (with local=True) or a remote URL. Under the hood, DeepEval base64-encodes local images before sending them to whichever multi-modal judge model you configure, so you don't need to manage encoding yourself. This is the detail that trips people up first: if you pass a local path without setting local=True, DeepEval will try to treat it as a URL and fail silently or throw a fetch error, so double-check that flag whenever a test unexpectedly errors out on image loading.

Metrics Built for Images, Not Just Text

Once you have MLLMTestCase objects, the next question is which metric actually measures what you care about. DeepEval ships several multi-modal metrics, and picking the wrong one is the most common way teams get misleadingly rosy scores.

Image hallucination detection is usually the first metric worth adopting, because it directly answers "did the model make something up about the image." Under the hood, this works like DeepEval's text hallucination metric: the judge model is shown the image plus the claim, and asked to verify each factual assertion against what's actually visible.

from deepeval.metrics import MultimodalGEval
from deepeval.test_case import MLLMTestCaseParams

hallucination_metric = MultimodalGEval(
    name="Image Faithfulness",
    criteria=(
        "Determine whether the 'actual_output' contains any claims about "
        "the image in 'input' that are not visually supported by it. "
        "Penalize invented numbers, colors, labels, or objects that are "
        "not actually present in the image."
    ),
    evaluation_params=[
        MLLMTestCaseParams.INPUT,
        MLLMTestCaseParams.ACTUAL_OUTPUT,
    ],
    threshold=0.7,
)

hallucination_metric.measure(test_case)
print(hallucination_metric.score, hallucination_metric.reason)

MultimodalGEval is the multi-modal sibling of DeepEval's popular GEval metric — you write a plain-English criteria string, and DeepEval compiles it into a chain-of-thought evaluation that a judge LLM runs against your test case, images included. This is the workhorse metric for most multi-modal testing because you rarely need something exotic; you need a judge that can actually see the picture and reason about specific, nameable failure modes: wrong counts, wrong colors, invented text, misread charts.

Beyond MultimodalGEval, DeepEval also provides more specialized multi-modal metrics for image generation and editing tasks — things like image coherence (does the generated image match the prompt) and image editing fidelity (did the edit only change what it was supposed to change, leaving the rest of the image intact). If your app generates or edits images rather than just describing them, those are worth reaching for instead of forcing a description-style criteria onto a generation task.

Building a Faithfulness Check for Chart and Screenshot QA

A concrete scenario that comes up constantly: an internal analytics agent that reads a chart screenshot and writes a summary. This is exactly the kind of feature where a model will confidently report a trend that isn't there, and nobody notices until a stakeholder asks "wait, where does it say 18%?"

Here's a fuller example that builds a small test suite around this use case:

from deepeval import assert_test
from deepeval.test_case import MLLMTestCase, MLLMImage, MLLMTestCaseParams
from deepeval.metrics import MultimodalGEval

def summarize_chart(image_path: str) -> str:
    # your actual VLM call goes here
    return call_vision_model(image_path)

chart_faithfulness = MultimodalGEval(
    name="Chart Faithfulness",
    criteria=(
        "Check every number, percentage, and trend direction claimed in "
        "'actual_output' against the chart shown in 'input'. Fail if any "
        "figure is fabricated, rounded misleadingly, or if a trend "
        "direction (up/down/flat) contradicts the chart."
    ),
    evaluation_params=[
        MLLMTestCaseParams.INPUT,
        MLLMTestCaseParams.ACTUAL_OUTPUT,
    ],
    threshold=0.8,
)

test_cases = []
for chart_path in ["charts/q1_revenue.png", "charts/q2_signups.png"]:
    output = summarize_chart(chart_path)
    test_cases.append(
        MLLMTestCase(
            input=["Summarize the key trend in this chart.", MLLMImage(url=chart_path, local=True)],
            actual_output=[output],
        )
    )

for tc in test_cases:
    assert_test(tc, [chart_faithfulness])

Notice that the criteria string does real work here — it's specific about the failure modes ("fabricated," "rounded misleadingly," "trend direction contradicts"). A vague criteria like "is the summary good" will produce a judge that scores fluent-sounding hallucinations highly, because fluency is not faithfulness. The more your criteria names concrete, checkable failure modes, the more reliable the judge's verdict becomes — this is true for GEval generally, but it's especially important in multi-modal evaluation because the judge has more surface area (visual details, not just text) to get lazy about.

Handling Multiple Images and Multi-Turn Visual Context

Real applications rarely deal with a single static image. A UI-testing agent screenshots before and after a click; a document-QA app might reference three pages of a PDF converted to images; a retail app might compare a user's uploaded photo against several product images to check fit.

MLLMTestCase handles this naturally since input and actual_output are lists — you just include more MLLMImage entries in order. The judge model receives them in sequence, so ordering matters: put images in the same order a human would need to see them to answer the question.

test_case = MLLMTestCase(
    input=[
        "Here is the product the customer wants:",
        MLLMImage(url="products/red_jacket.png", local=True),
        "Here are three jackets currently in stock. Which is the closest match?",
        MLLMImage(url="stock/jacket_a.png", local=True),
        MLLMImage(url="stock/jacket_b.png", local=True),
        MLLMImage(url="stock/jacket_c.png", local=True),
    ],
    actual_output=[
        "Jacket B is the closest match — same red shade and collar style, though the zipper color differs.",
    ],
)

For agentic, multi-turn visual workflows — think a browser agent that takes a screenshot, clicks something, takes another screenshot, and reasons about the diff across several steps — you'll often want to evaluate each step's MLLMTestCase independently rather than cramming an entire session into one test case. Judge models degrade in accuracy as you stack more images into a single context, so smaller, focused test cases per step tend to produce more trustworthy scores than one giant multi-image test case covering an entire session.

Choosing and Configuring a Multi-Modal Judge Model

Metrics are only as good as the judge model evaluating them, and this matters more in multi-modal testing than in text-only testing because not every model you'd normally reach for can actually process images. DeepEval lets you plug in any multi-modal-capable model as the evaluator, but you need to explicitly configure one — it won't silently fall back to a text-only model and pretend to look at the image.

from deepeval.models import GPTModel

judge_model = GPTModel(model="gpt-4o")

hallucination_metric = MultimodalGEval(
    name="Image Faithfulness",
    criteria="Flag any claim about the image that isn't visually supported.",
    evaluation_params=[MLLMTestCaseParams.INPUT, MLLMTestCaseParams.ACTUAL_OUTPUT],
    model=judge_model,
    threshold=0.7,
)

A few practical notes worth internalizing here:

• Match judge capability to task difficulty. Reading a bar chart with three bars is easy for most vision models; reading a dense financial table screenshot with small fonts is not. If your judge model can't reliably read the image itself, its verdicts are noise regardless of how good your criteria string is. Spot-check a handful of judge verdicts manually before trusting the metric at scale.
• Watch image resolution and compression. Downscaled or heavily compressed screenshots can make small text illegible to the judge model even when a human could read it fine on the original. If your CI pipeline compresses screenshots before storing them as test fixtures, verify the judge model can still read the relevant details post-compression.
• Keep judge and production model separate where possible. Using the exact same model as both the system under test and the judge can mask shared blind spots — if a model consistently misreads a particular chart style, it may also fail to notice that misreading when acting as its own judge.

Running Multi-Modal Evals in CI with Datasets

Individual test cases are useful for debugging one failure, but the real value of DeepEval shows up when you assemble a dataset of multi-modal test cases and run them as a regression suite, the same way you'd run unit tests before merging a PR.

from deepeval.dataset import EvaluationDataset
from deepeval import evaluate

dataset = EvaluationDataset(test_cases=test_cases)

evaluate(
    test_cases=dataset.test_cases,
    metrics=[chart_faithfulness],
)

Wire this into a pytest file and it becomes part of your normal CI gate:

import pytest
from deepeval import assert_test

@pytest.mark.parametrize("test_case", test_cases)
def test_chart_summaries(test_case):
    assert_test(test_case, [chart_faithfulness])

deepeval test run test_chart_eval.py

A pattern worth adopting early: store your reference images (screenshots, charts, product photos) as versioned fixtures alongside your test code, not as ephemeral files generated at test time. Multi-modal test failures are much harder to debug from a score and a reason string alone — you want to be able to open the exact image the judge saw and eyeball it yourself when a score looks wrong. Treat your image fixtures the same way you'd treat golden files in snapshot testing: committed, reviewed, and stable across runs unless intentionally updated.

Common Failure Patterns to Design Tests Around

After running multi-modal evals against a few different vision-language features, certain failure patterns show up repeatedly, and it's worth writing criteria strings that specifically target them rather than relying on a generic "is this accurate" prompt.

1. Numeric hallucination in charts and tables. Models frequently invent precise-sounding numbers ("increased by 23.4%") that aren't actually derivable from the image. Criteria should explicitly require every number to be traceable to a labeled value or axis in the image.
2. Object count errors. Ask a VLM "how many people are in this photo" and it will often be off by one, especially with partial occlusion. If counting matters to your use case, add a dedicated counting-accuracy criteria rather than folding it into a general faithfulness check.
3. Color and spatial relationship mistakes. "The red button is on the left" — models get left/right and color-under-lighting wrong more often than you'd expect, particularly with screenshots that have similar-colored UI elements.
4. Text-in-image misreads (OCR-adjacent errors). Small or stylized text in screenshots and photos is a common source of subtly wrong quotes. If your app extracts text from images, consider comparing extracted text against ground truth separately from your faithfulness metric, since OCR-style errors and reasoning-style hallucinations need different fixes.
5. Overconfidence on low-quality images. Blurry, dark, or heavily cropped images should produce hedged answers ("it's unclear from the image whether..."), not confident wrong ones. You can write a criteria specifically penalizing unwarranted confidence when the image itself is ambiguous, which is a subtler but very real failure mode worth testing for directly.

Building a small regression set that deliberately includes a few images from each of these categories, alongside your normal "happy path" images, will surface far more real issues than testing only against clean, unambiguous images.

Getting Started Without Overengineering It

You don't need a fully automated CI pipeline on day one to get value out of this. A reasonable rollout path looks like:

• Start with five to ten representative images from your actual product surface — real screenshots or photos, not synthetic test images, since real data surfaces real failure modes.
• Write one MultimodalGEval metric with a specific, narrow criteria string targeting your app's most likely failure mode (chart numbers, object counts, whatever matters most for your feature).
• Run it manually against your current model outputs and read the reason field for every score, not just the number — this is where you'll catch that your criteria is too vague or your judge model can't actually see what you think it can.
• Once the metric's verdicts match your own judgment on a handful of cases, wire it into assert_test inside a pytest file and add it to CI so future model or prompt changes get caught automatically.

Multi-modal evaluation is newer territory than text evaluation, and it's tempting to either skip it entirely ("we'll just eyeball outputs") or overbuild it with a dozen metrics before you've validated a single one works well against your data. Neither serves you. Start with the one failure mode that would actually embarrass you in production, write a metric for exactly that, and expand from there.

If you want a structured, hands-on path through this rather than piecing it together from documentation, the DeepEval Tutorial course on teachyou.ai walks through building a complete multi-modal evaluation suite from scratch — including judge model selection, dataset construction, and wiring everything into a CI pipeline — with real vision-language test cases you can adapt directly to your own product.