LLM Eval

Evaluating Hallucination Rate: Detection Techniques Compared

Pramod Dutta · Jun 7, 2026 · 12 min read

Ship an LLM feature to production and within a week someone will forward you a screenshot: the model confidently cited a court case that never happened, invented a changelog entry, or summarized a document with a statistic that appears nowhere in the source. Your first instinct is to patch the prompt. Your second instinct, if you've been burned before, is to ask a harder question: how do I actually measure how often this happens, across thousands of outputs, without reading every single one by hand? That's the real problem behind "evaluating hallucination rate," and it turns out to be less about picking one metric and more about picking the right detector for the right failure mode. This piece walks through the detection techniques that actually get used in production evals, where each one breaks, and how to combine them into a pipeline you can trust.

What "hallucination" actually means for evaluation purposes

Before comparing detection techniques, it helps to split "hallucination" into categories, because a single blanket metric will systematically miss some of them.

Intrinsic hallucination: the output contradicts the provided source/context. Example: your RAG system retrieves a support doc that says a refund window is 30 days, but the model tells the user it's 60 days.
Extrinsic hallucination: the output adds information that isn't in the source and can't be verified from it, even if it happens to be true. Example: the model appends "this policy was introduced in 2019" when the source doc says nothing about when the policy started.
Closed-domain hallucination: happens in tasks with a fixed reference, like summarization or translation, where you have ground truth to compare against.
Open-domain hallucination: happens in open-ended generation, like chatbots or agents, where there's no single correct answer, only world facts and plausibility.

Most teams conflate these when they say "hallucination rate," and it causes real damage: a detector tuned for intrinsic contradiction (like an NLI-based faithfulness checker) will happily give a passing score to a fluent, extrinsic fabrication that doesn't technically contradict anything, because it never claimed to check for made-up facts, only decoherence. When you build an eval pipeline, decide up front which categories you're scoring and pick a detector suited to each, or you'll get a number that looks stable but hides real failures.

Reference-based detection: when you have ground truth

The simplest case is closed-domain generation: summarization, structured extraction, or question answering over a known document. Here you can lean on classic NLP metrics repurposed for factuality.

N-gram overlap metrics (ROUGE, BLEU) are the oldest tool in the box and the least useful for hallucination specifically. They measure surface overlap with a reference, not factual consistency. A summary can score high on ROUGE while inventing a number, and it can score low while being perfectly faithful just because it paraphrased well. Don't use these as your hallucination signal; use them only as a sanity check on fluency.

Natural Language Inference (NLI) based faithfulness scoring is far more useful. The idea: treat each claim in the generated output as a "hypothesis" and the source document as the "premise." Run an NLI model and check whether the premise entails the hypothesis. If the source contradicts or fails to support the claim, flag it as unfaithful.

from transformers import pipeline

nli = pipeline("text-classification", model="microsoft/deberta-large-mnli")

def faithfulness_score(source: str, claim: str) -> dict:
    """Score whether `source` entails `claim` using NLI.
    Returns label (entailment/neutral/contradiction) and confidence.
    """
    # NLI models expect premise + hypothesis, joined with a separator
    result = nli(f"{source} [SEP] {claim}")[0]
    return {"label": result["label"], "score": result["score"]}

source_doc = (
    "Our refund policy allows returns within 30 days of purchase, "
    "provided the item is unused and in original packaging."
)
generated_claim = "You can return the item within 60 days for a full refund."

print(faithfulness_score(source_doc, generated_claim))
# Expect: {'label': 'CONTRADICTION', 'score': 0.91} or similar

To use this at scale, you first decompose the generated output into atomic claims (one factual assertion per sentence, roughly), then run each claim against the retrieved source chunks, then aggregate: percentage of claims marked "contradiction" or "neutral" (unsupported) becomes your hallucination rate for that response. This claim-decomposition step matters more than the NLI model choice — if you skip it and feed whole paragraphs into the NLI model, you get noisy, low-resolution scores because one contradicted sentence buried in five correct ones won't move the needle much.

Question-answering based consistency (QAG/QAFactEval-style) flips the approach: generate questions from the output, answer them using the source, and check whether the answers match. If the summary says "the CEO resigned in March," you generate "When did the CEO resign?", answer it from the source ("June"), and flag the mismatch. This tends to be more interpretable for humans reviewing the eval output than a raw NLI score, because you get a concrete question/answer pair to inspect, not just a probability.

Reference-free detection: the harder, more common case

Most production LLM systems don't have a clean reference to check against — chatbots, agents, open-ended Q&A. Here you need detectors that work without ground truth.

Self-consistency / sampling-based detection (SelfCheckGPT-style) is the workhorse technique here. The intuition: if a model actually "knows" a fact, it will state it consistently across multiple independent samples. If it's hallucinating, the samples will diverge, because the model is essentially guessing each time.

import openai

client = openai.OpenAI()

def sample_responses(prompt: str, n: int = 5, temperature: float = 0.9) -> list[str]:
    responses = []
    for _ in range(n):
        completion = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": prompt}],
            temperature=temperature,
        )
        responses.append(completion.choices[0].message.content)
    return responses

def consistency_check(main_response: str, samples: list[str]) -> float:
    """Rough consistency score: ask a judge model whether each sample
    supports or contradicts the main response's key claims.
    Returns fraction of samples that are consistent (0-1, higher = more trustworthy).
    """
    consistent = 0
    for sample in samples:
        verdict = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{
                "role": "user",
                "content": (
                    f"Response A: {main_response}\n\nResponse B: {sample}\n\n"
                    "Do these two responses agree on the key facts? "
                    "Answer only 'yes' or 'no'."
                ),
            }],
            temperature=0,
        ).choices[0].message.content.strip().lower()
        if verdict.startswith("yes"):
            consistent += 1
    return consistent / len(samples)

main = sample_responses("Who founded the company Stripe and in what year?", n=1)[0]
others = sample_responses("Who founded the company Stripe and in what year?", n=5)
print(consistency_check(main, others))

This works well as a black-box hallucination detector because it needs no access to model internals or a labeled reference — just API access and a budget for extra samples. The tradeoff is cost: checking one response now costs 5-6x the tokens. In practice, teams run this as a sampling-based audit on a subset of production traffic (say, 2-5%) rather than on every request.

Token-level uncertainty / logprob-based detection works if you have access to the model's log-probabilities (open-weight models, or APIs that expose logprobs). The premise: hallucinated tokens tend to have lower model confidence, especially at "decision points" where the model is inventing a specific entity, number, or date rather than following a well-supported pattern.

import math

def flag_low_confidence_spans(tokens: list[str], logprobs: list[float], threshold: float = -3.0):
    """Flag contiguous spans of tokens where average logprob drops below threshold.
    Low logprob often correlates with the model 'guessing' rather than recalling.
    """
    flagged = []
    window = 3
    for i in range(len(tokens) - window + 1):
        span_logprobs = logprobs[i:i + window]
        avg = sum(span_logprobs) / window
        if avg < threshold:
            flagged.append({
                "span": tokens[i:i + window],
                "avg_logprob": avg,
                "perplexity": math.exp(-avg),
            })
    return flagged

This is cheap and fast since it requires no extra API calls, but it's a proxy signal, not a factuality check. A model can be highly confident and still wrong (this is common with dates, statistics, and less popular entities memorized incorrectly during pretraining), and it can be uncertain about something that's actually true but rare. Use it as a triage filter to prioritize which outputs a human or a stronger detector should look at, not as the final verdict.

Retrieval-augmented verification treats every claim in the output as something to fact-check against an external source, live, regardless of whether the original generation used retrieval. You extract atomic claims, run a search or knowledge-base lookup for each, and score whether the claim is supported, refuted, or unverifiable.

def verify_claims(claims: list[str], retriever) -> list[dict]:
    """For each atomic claim, retrieve supporting evidence and classify.
    `retriever` is any callable that returns top-k passages for a query.
    """
    results = []
    for claim in claims:
        evidence = retriever(claim, k=3)
        # Reuse the NLI-based faithfulness check against retrieved evidence
        verdicts = [faithfulness_score(passage, claim) for passage in evidence]
        best = max(verdicts, key=lambda v: v["score"])
        results.append({"claim": claim, "verdict": best["label"], "evidence": evidence})
    return results

This is the most rigorous reference-free technique because it grounds every claim in something external rather than the model's own consistency, but it's also the most expensive and infrastructure-heavy: you need a claim extractor, a retriever with decent recall, and a way to handle claims that are genuinely unverifiable (opinions, predictions, subjective statements) without flagging them as hallucinations.

LLM-as-a-Judge for hallucination scoring

The technique that's eaten the most market share in the last two years is simply prompting a strong model to judge another model's output for factual accuracy. It's fast to set up, doesn't require training a classifier, and handles nuance that rule-based methods miss.

JUDGE_PROMPT = """You are evaluating an AI response for factual hallucination.

SOURCE CONTEXT:
{context}

AI RESPONSE:
{response}

Instructions:
1. List every factual claim made in the AI response.
2. For each claim, mark it as SUPPORTED (stated in context), CONTRADICTED
   (context says something different), or UNSUPPORTED (not mentioned in context at all).
3. Compute a hallucination score: (CONTRADICTED + UNSUPPORTED) / total claims.

Respond in this exact format:
CLAIMS:
- <claim>: <SUPPORTED|CONTRADICTED|UNSUPPORTED>
SCORE: <float between 0 and 1>
"""

def judge_hallucination(context: str, response: str, judge_model="gpt-4o") -> dict:
    completion = client.chat.completions.create(
        model=judge_model,
        messages=[{
            "role": "user",
            "content": JUDGE_PROMPT.format(context=context, response=response),
        }],
        temperature=0,
    )
    return completion.choices[0].message.content

The appeal is obvious: one prompt, no training data, works on open-domain and closed-domain cases alike, and it produces a human-readable rationale you can spot-check. The failure modes are equally real, though. Judge models have their own blind spots — they're worse at catching hallucinations in domains they don't understand well (legal citations, medical dosages, niche technical specs), they can be biased toward longer or more confident-sounding responses, and they occasionally hallucinate their own verdict, claiming a fact is "unsupported" when it's actually stated in the context but phrased differently. This is why judge-based scoring needs calibration against a human-labeled sample before you trust it at scale, and why the strongest pipelines don't rely on it alone.

Building a composite hallucination rate metric

None of these techniques should run alone in a serious eval pipeline. Here's a pattern that combines cheap filters with expensive verification, which keeps cost sane while catching more failure modes than any single method:

def compute_hallucination_rate(dataset: list[dict], retriever=None) -> dict:
    """
    dataset: list of {"context": str, "response": str, "logprobs": list, "tokens": list}
    Returns aggregate hallucination rate plus per-method breakdown.
    """
    results = {"nli": [], "logprob_flags": [], "judge": []}

    for item in dataset:
        # Stage 1: cheap logprob triage, always run
        flags = flag_low_confidence_spans(item["tokens"], item["logprobs"])
        results["logprob_flags"].append(len(flags))

        # Stage 2: NLI faithfulness against retrieved/given context
        claims = split_into_claims(item["response"])  # your own sentence splitter
        nli_scores = [faithfulness_score(item["context"], c) for c in claims]
        unfaithful = sum(1 for s in nli_scores if s["label"] != "ENTAILMENT")
        results["nli"].append(unfaithful / max(len(claims), 1))

        # Stage 3: only escalate to the expensive judge when stages 1-2 disagree
        # or flag something, to control cost
        if unfaithful > 0 or len(flags) > 0:
            verdict = judge_hallucination(item["context"], item["response"])
            results["judge"].append(verdict)

    avg_nli_rate = sum(results["nli"]) / len(results["nli"])
    return {
        "aggregate_hallucination_rate": avg_nli_rate,
        "escalated_to_judge": len(results["judge"]),
        "total_examples": len(dataset),
    }

The escalation pattern here (cheap detector filters, expensive detector confirms) is the single highest-leverage design decision in a hallucination eval pipeline. Running an LLM judge on every response in a 50,000-example eval set is slow and costly; running it only on the 8-12% that a logprob or NLI filter flagged as suspicious gets you nearly the same detection quality at a fraction of the cost.

Common measurement pitfalls that skew your numbers

A few mistakes show up repeatedly in teams building their first hallucination eval, and each one silently distorts the reported rate.

Treating "unsupported" and "contradicted" as the same severity. A claim that's simply not mentioned in the source (extrinsic) is often lower-risk than one that actively contradicts the source (intrinsic). Report them separately; averaging them into one number hides which failure mode is actually growing.
Evaluating on a static, stale test set. Hallucination rate drifts as you change prompts, swap models, or update retrieval indexes. A number measured once and never rechecked becomes actively misleading within a few weeks of any pipeline change.
Ignoring claim granularity. Scoring a whole paragraph as one unit instead of decomposing into atomic claims dramatically understates the rate, because one wrong sentence in a mostly-correct paragraph barely moves a paragraph-level score.
Using a judge model that's weaker than the model being evaluated. If your production model is more capable than your judge, the judge will miss subtle fabrications it can't itself reason about. As a rule of thumb, the judge should be at least as strong as, ideally stronger than, the model under test.
No human calibration step. Every automated detector — NLI, self-consistency, logprobs, LLM judges — needs to be checked against a few hundred human-labeled examples before you trust its output. Skipping this step means you're reporting a number with unknown error bars.

Putting it into a repeatable eval workflow

A hallucination rate number is only useful if it's produced the same way every time and tracked over time, not computed once for a slide deck. A workable workflow looks like this: define your claim decomposition method and freeze it, pick one or two detectors per hallucination category (intrinsic vs extrinsic, closed vs open-domain), calibrate each detector against roughly 200-300 human-labeled examples and report agreement (Cohen's kappa is a reasonable choice here), then wire the pipeline into CI or a nightly job so every prompt change, model swap, or retrieval update produces a fresh number you can diff against the last run. Treat the composite score less like a leaderboard metric and more like a regression test: the question isn't "is 4% hallucination rate good," it's "did this change make the rate go up."

The techniques above (NLI-based faithfulness, self-consistency sampling, logprob triage, retrieval verification) each catch a different slice of the problem, and none of them is sufficient alone. What ties them together in almost every production pipeline today is a judge model doing the final adjudication, because it's the only method flexible enough to reason about nuance, partial truths, and context that rule-based scoring can't handle. Getting comfortable designing, calibrating, and stress-testing an LLM-as-a-Judge step, so that it's a rigorous evaluator rather than a rubber stamp, is quickly becoming one of the core skills for anyone shipping LLM products at scale.