Embedding-Similarity Metrics for LLM Evaluation

Embedding similarity metrics score an LLM output by turning both the output and a reference into vectors and measuring the angle between them, so a paraphrase that keeps the meaning still scores high even when the exact words differ. That is the whole point: they measure semantic closeness instead of token overlap, which is why they replaced BLEU and ROUGE for most generation tasks. This guide shows you how embedding similarity metrics work, how to compute them with real code, where they break, and how to calibrate a threshold you can actually ship.

Why embedding similarity metrics beat word-overlap scores

If you have ever watched a correct answer get a low BLEU score, you already understand the problem. BLEU and ROUGE count overlapping n-grams. The reference says "The capital of France is Paris" and the model says "Paris is France's capital." Every meaningful word is there, the answer is perfect, and n-gram overlap collapses because the word order and joining tokens changed. Word-overlap metrics were built for machine translation in an era when outputs were expected to track a reference closely. LLM outputs do not behave that way.

Embedding similarity metrics fix this by moving the comparison into vector space. An embedding model maps text to a dense vector, typically a few hundred to a few thousand dimensions, where distance encodes meaning. Two texts that mean the same thing land near each other regardless of surface wording. You compute a single number, usually cosine similarity, and that number is your score.

The practical payoff for engineers building evaluation harnesses:

• Paraphrase tolerance. Reworded correct answers score high instead of getting penalized.
• Language and format robustness. Bullet points versus prose, active versus passive, contractions versus full forms, none of these tank the score the way they tank ROUGE.
• Cheap to run. One embedding call per text plus a dot product. No judge model, no rubric prompt, no per-token generation cost.
• Deterministic. The same inputs give the same score every run, which word-level LLM-as-judge metrics cannot promise.

The tradeoff is that embedding similarity metrics measure topical and semantic proximity, not correctness. "The capital of France is Paris" and "The capital of France is Lyon" are embarrassingly close in vector space because they share almost all their structure and topic. Keep that failure mode in mind; the last sections of this article are built around it.

The core math: cosine, dot product, and Euclidean distance

Three distance functions show up constantly. You should know exactly what each one does before you pick one.

Cosine similarity measures the angle between two vectors, ignoring their magnitude. It ranges from -1 to 1, though for modern text embeddings you will almost never see negatives, so in practice it sits between roughly 0 and 1. This is the default for embedding similarity metrics because meaning is encoded in direction, not length.

Dot product multiplies the vectors component-wise and sums. If your embeddings are normalized to unit length, the dot product equals cosine similarity exactly. Many embedding APIs already return normalized vectors, which is why dot product and cosine are often used interchangeably. Check your provider's docs before assuming.

Euclidean distance (L2) measures straight-line distance between the vector tips. It is sensitive to magnitude, so on unnormalized embeddings it can rank things differently from cosine. For normalized vectors, L2 distance and cosine similarity are monotonically related, meaning they rank pairs in the same order even though the numbers differ.

Here is all three, dependency-free except NumPy, so you can see there is no magic:

import numpy as np

def cosine_similarity(a, b):
    a = np.asarray(a, dtype=np.float64)
    b = np.asarray(b, dtype=np.float64)
    denom = np.linalg.norm(a) * np.linalg.norm(b)
    if denom == 0.0:
        return 0.0
    return float(np.dot(a, b) / denom)

def dot_product(a, b):
    return float(np.dot(np.asarray(a), np.asarray(b)))

def euclidean_distance(a, b):
    a = np.asarray(a, dtype=np.float64)
    b = np.asarray(b, dtype=np.float64)
    return float(np.linalg.norm(a - b))

A note that saves debugging hours: if your vectors are already unit-normalized, cosine_similarity(a, b) and dot_product(a, b) return the same value, and you can drop the normalization in cosine to save cycles. To normalize once, up front:

def normalize(v):
    v = np.asarray(v, dtype=np.float64)
    n = np.linalg.norm(v)
    return v / n if n else v

For a whole matrix of embeddings, do not loop. Normalize the rows, then a single matrix multiply gives you every pairwise cosine at once:

def cosine_matrix(embeddings):
    m = np.asarray(embeddings, dtype=np.float64)
    norms = np.linalg.norm(m, axis=1, keepdims=True)
    norms[norms == 0] = 1.0
    unit = m / norms
    return unit @ unit.T

Computing embedding similarity metrics end to end

The full loop is: get an embedding for the candidate, get an embedding for the reference, compute cosine, compare to a threshold. Below is a runnable version using an embedding endpoint. Swap the client for whichever provider you use; the shape is identical across OpenAI, Cohere, Voyage, and open-source models served through sentence-transformers.

Using an API-based embedding model:

from openai import OpenAI

client = OpenAI()

def embed(texts, model="text-embedding-3-small"):
    if isinstance(texts, str):
        texts = [texts]
    resp = client.embeddings.create(model=model, input=texts)
    return [d.embedding for d in resp.data]

def semantic_score(candidate, reference):
    c_vec, r_vec = embed([candidate, reference])
    return cosine_similarity(c_vec, r_vec)

print(semantic_score(
    "Paris is the capital of France.",
    "The capital of France is Paris."
))

Batch both texts in a single embed call. It halves your request count and keeps latency down when you are scoring thousands of rows.

If you want a fully local, offline metric with no API cost, sentence-transformers is the standard tool. It ships the encoder and a cosine helper:

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")

def local_score(candidate, reference):
    emb = model.encode([candidate, reference], normalize_embeddings=True)
    return float(util.cos_sim(emb[0], emb[1]))

The all-MiniLM-L6-v2 model produces 384-dimensional vectors, runs on CPU, and is fast enough to score a full regression set in seconds. Larger encoders give slightly better semantic resolution at higher compute cost. For most evaluation harnesses the small model is the right starting point because you care about relative ordering and a stable threshold, not the last decimal of accuracy.

To score a whole test set and get a pass rate, batch everything and threshold once:

def evaluate_dataset(rows, threshold=0.80):
    candidates = [r["output"] for r in rows]
    references = [r["reference"] for r in rows]
    c_emb = model.encode(candidates, normalize_embeddings=True)
    r_emb = model.encode(references, normalize_embeddings=True)
    scores = (c_emb * r_emb).sum(axis=1)  # row-wise cosine, normalized inputs
    passes = scores >= threshold
    return {
        "mean_score": float(scores.mean()),
        "pass_rate": float(passes.mean()),
        "scores": scores.tolist(),
    }

Because the inputs are normalized, the row-wise product-and-sum is exact cosine, no division needed. That is the fastest correct way to score a batch.

Choosing an embedding model for evaluation

The metric is only as good as the encoder underneath it. A few decision points that matter more than which brand you pick:

• Symmetric versus asymmetric. Some models are trained for symmetric similarity (two texts of similar length and role, like answer versus reference). Others are trained asymmetrically for retrieval (short query versus long document). For output-versus-reference scoring you want a symmetric similarity model. Using a retrieval-tuned model with a query prefix on both sides will skew your numbers.
• Dimensionality. Higher dimensions capture more nuance but cost more to store and compute. Some newer models support shortening the vector (Matryoshka-style truncation) so you can trade a little accuracy for a lot of speed. If your provider supports a dimensions parameter, test whether the shorter vector changes your pass rate before paying for the full length.
• Domain fit. General-purpose encoders handle everyday prose well and struggle with dense jargon: legal clauses, medical coding, source code. If your outputs are code, use a code-aware embedding model or your similarity scores will be noisy.
• Consistency over absolute quality. Whatever you pick, freeze it. If you change the embedding model, every historical score becomes incomparable and your threshold is invalidated. Pin the model name and version in your eval config the same way you pin a dependency.

Do not mix embedding models within one metric. The candidate and the reference must go through the same encoder or the cosine number is meaningless.

Semantic similarity beyond a single reference

Real evaluation rarely has one golden answer. Two patterns extend embedding similarity metrics to messier ground.

Multi-reference scoring. When several answers are all acceptable, embed each reference and take the maximum cosine against the candidate. The candidate should be close to at least one acceptable answer:

def multi_reference_score(candidate, references):
    c_vec = embed([candidate])[0]
    r_vecs = embed(references)
    sims = [cosine_similarity(c_vec, r) for r in r_vecs]
    return max(sims)

Reference-free relevance. Sometimes you have no reference at all, only the input prompt and the output. You can still compute how on-topic the answer is by scoring the output against the question. This does not measure correctness, only relevance, and it is a genuinely useful guardrail for catching non-sequitur answers and topic drift:

def answer_relevance(question, answer):
    q_vec, a_vec = embed([question, answer])
    return cosine_similarity(q_vec, a_vec)

RAG faithfulness by chunks. For retrieval-augmented generation, embedding similarity metrics help you check whether the answer stays grounded in the retrieved context. Split the answer into sentences, embed each one, and measure the highest similarity against the retrieved chunks. A sentence that is far from every chunk is a candidate hallucination:

def grounding_scores(answer_sentences, context_chunks):
    a_emb = model.encode(answer_sentences, normalize_embeddings=True)
    c_emb = model.encode(context_chunks, normalize_embeddings=True)
    sims = a_emb @ c_emb.T          # sentence x chunk cosine matrix
    return sims.max(axis=1)         # best-supporting chunk per sentence

This is a signal, not a verdict. A low score flags a sentence for review; it does not prove the sentence is wrong. Established RAG evaluation frameworks combine this kind of embedding signal with an LLM judge for the final call, and that layering is the right instinct.

Calibrating a threshold you can trust

A cosine score of 0.82 means nothing until you know what 0.82 means for your data and your encoder. Thresholds are not portable across models or domains. Calibrate.

Start with a small labeled set: 50 to 100 candidate-reference pairs, each marked pass or fail by a human. Compute the cosine for every pair, then sweep the threshold and pick the value that best separates your labels. This snippet finds the threshold that maximizes agreement with human judgment:

import numpy as np

def calibrate_threshold(scores, labels):
    scores = np.asarray(scores)
    labels = np.asarray(labels).astype(bool)
    best_t, best_acc = 0.5, 0.0
    for t in np.linspace(0.3, 0.95, 66):
        pred = scores >= t
        acc = (pred == labels).mean()
        if acc > best_acc:
            best_acc, best_t = acc, float(t)
    return best_t, best_acc

Accuracy is a blunt objective. If false negatives (rejecting good answers) and false positives (passing bad answers) have different costs, sweep for the F-score you care about or set the threshold at a fixed precision or recall on your labeled set instead. The mechanics are the same; only the objective changes.

Two habits that keep calibration honest:

• Recalibrate when anything upstream changes. New embedding model, new prompt template, new content domain, all of them can shift the score distribution. A threshold is a property of the whole pipeline, not a constant.
• Report the distribution, not just the mean. A mean cosine of 0.85 with everything clustered tightly is healthy. The same mean with a bimodal split, half at 0.95 and half at 0.75, means you have two populations and a single threshold will misjudge one of them.

Where embedding similarity metrics lie to you

These metrics are fast and stable, and they will confidently mislead you in specific, predictable ways. Know them before you trust a green dashboard.

Negation blindness. "The drug is safe for pregnant women" and "The drug is not safe for pregnant women" are almost identical in vector space. They differ by one word that inverts the entire meaning, and cosine similarity barely moves. For any domain where negation flips correctness, embedding similarity alone is dangerous.

Numeric and entity swaps. "Ship 500 units by Friday" versus "Ship 5000 units by Monday" scores high. The structure and topic dominate the vector; the specific numbers and dates contribute little. If exact values matter, pair the metric with a rule-based check that extracts and compares the numbers directly.

Fluent nonsense. A confidently written wrong answer that stays on topic scores well because it is topically close to the reference. Embedding similarity measures aboutness, not truth.

Length and verbosity effects. A padded answer that buries the correct fact in extra text can drift from a terse reference. Sometimes that penalizes a fine answer; sometimes it correctly flags waffle. Either way it is length sensitivity, not a correctness signal, so do not read too much into small gaps.

Style and tone insensitivity. If your task cares about tone, a polite refusal versus a rude one, embedding similarity will treat near-identical content as near-identical regardless of tone. Match the metric to what you actually need to measure.

The rule that follows from all of this: use embedding similarity metrics as a fast first-pass filter and a regression tripwire, not as the sole arbiter of correctness. They are excellent at catching outputs that wandered off topic and at flagging drift between model versions. For the pass or fail decision on anything where negation, numbers, or truth matter, layer an LLM-as-judge check or a deterministic rule on top of the ones the embedding metric passes. You get the throughput of a cheap metric with a guardrail against its blind spots.

A practical evaluation stack

Putting the pieces together, a pragmatic harness for a generation or RAG system looks like tiers, cheapest first:

1. Embedding similarity as the wide net. Score every output against its reference or context. Anything far below threshold is auto-flagged; you never spend a judge call on obvious failures.
2. Deterministic checks for the things embeddings miss. Regex or parsers for required numbers, dates, entities, and formats. Cheap, exact, and immune to negation blindness.
3. LLM-as-judge on the survivors. Reserve the expensive, slower judge for outputs that passed the first two tiers, where the remaining question is genuine quality rather than gross error.

This ordering matters because embedding similarity metrics are the only tier that scales to every output on every run without cost or latency pain. They earn their place at the front of the funnel precisely because they are cheap, deterministic, and paraphrase-tolerant. Let them do the volume work, and spend your judge budget where nuance actually lives.

Track the mean and the full score distribution over time as a regression signal. When you ship a new prompt or swap a model, a drop in mean cosine against a frozen reference set tells you something changed before any user complains. That early-warning use is where embedding similarity metrics quietly earn their keep in production.

FAQ

Are embedding similarity metrics better than BLEU and ROUGE? For most LLM generation tasks, yes, because they tolerate paraphrase and reward meaning over exact word overlap. BLEU and ROUGE still have a place when you genuinely need surface-form fidelity, like checking that a required phrase appears verbatim. For open-ended answers where many wordings are correct, n-gram overlap punishes good outputs and embedding similarity does not.

What cosine similarity threshold means "correct"? There is no universal number. A threshold is a property of your embedding model plus your data, so 0.80 on one encoder and domain is not the same bar as 0.80 on another. Calibrate against a small human-labeled set, pick the threshold that best matches those labels, and recalibrate whenever you change the model, prompt, or domain.

Cosine similarity or Euclidean distance for text embeddings? Cosine similarity is the default because meaning in text embeddings is carried by direction, not magnitude. If your vectors are unit-normalized, Euclidean distance ranks pairs the same way cosine does, so the choice is cosmetic. On unnormalized vectors, prefer cosine unless you have a specific reason to care about magnitude.

Can I use embedding similarity to detect hallucinations in RAG? As a signal, yes. Embedding each answer sentence and measuring its best similarity against the retrieved chunks flags sentences that are not grounded in the context. It is a tripwire, not a verdict, because a low score means unsupported, not necessarily false. Production RAG evaluation pairs this signal with an LLM judge for the final grounding call.

Do I need a paid embedding API, or can I run this locally? You can run it fully locally. Libraries like sentence-transformers ship compact encoders such as all-MiniLM-L6-v2 that run on CPU and score a regression set in seconds with no API cost. Hosted embedding models give somewhat sharper semantic resolution, which matters more for retrieval than for a well-calibrated pass or fail metric. Start local, measure, and only reach for a paid model if your threshold cannot separate good from bad answers.

Why did my perfect answer get a low similarity score? Usually one of three causes. The reference and candidate went through different embedding models, so the vectors are not comparable. The encoder is retrieval-tuned rather than symmetric, skewing same-role comparisons. Or the answer is much longer or shorter than the reference and length sensitivity pulled the score down. Check the encoder first; a mismatched or wrong-type embedding model is the most common culprit.

How many labeled examples do I need to trust a threshold? Fifty to a hundred human-labeled pairs is enough to set a defensible starting threshold and see the score distribution clearly. More is better for stability, but the bigger risk is staleness, not sample size: a threshold calibrated on last quarter's data and this quarter's new prompt is the thing that quietly rots. Recalibrate on pipeline changes rather than chasing a huge one-time labeling effort.