Embedding Dimensionality Reduction with Matryoshka Representation Learning

Embedding dimensionality reduction is the practice of shrinking a vector's dimension count, say from 1536 down to 256, while keeping enough semantic information for search and retrieval to still work well. The most reliable modern approach is Matryoshka Representation Learning (MRL), a training technique that produces embeddings you can truncate to any smaller size without retraining or running a separate reduction step. If you're running a RAG pipeline or a vector search index and your storage bill or query latency is climbing, this is usually the first lever to pull before you touch your chunking strategy or your retriever.

This article covers why raw embedding dimension matters for cost and speed, how Matryoshka embeddings work under the hood, how to actually truncate and re-normalize vectors correctly, how this compares to PCA and other classic dimensionality reduction methods, and how to measure whether you lost meaningful retrieval quality.

Why embedding dimensionality matters for cost and speed

Every dimension in an embedding vector costs you three times: once in RAM or disk to store it, once in network bandwidth to move it, and once in CPU or GPU cycles to compute similarity against it. A 1536-dimension float32 vector takes 6 KB. Multiply that by 10 million documents and you're at 60 GB just for the raw vectors, before index overhead from something like HNSW, which commonly adds 1.5x to 3x on top depending on graph parameters.

Cosine similarity and dot product both scale linearly with dimension count. Halve the dimensions, and you roughly halve the arithmetic per comparison. For approximate nearest neighbor (ANN) indexes, lower dimensionality also means smaller graph nodes, better cache locality, and faster index builds. None of this is news, engineers have used PCA, random projections, and autoencoders for years to shrink embeddings. What's changed is that leading embedding providers now train models specifically so you can truncate their output vectors directly, with no separate reduction model to train, host, or version.

What Matryoshka Representation Learning actually does

MRL, introduced by researchers at Google and the University of Washington, changes how an embedding model is trained, not how you use it afterward. During training, the loss function is applied not just to the full embedding vector but also to several nested prefixes of it, say the first 64, 128, 256, 512, and full 1536 dimensions. This forces the model to pack the most important, most general semantic information into the earliest dimensions, with each additional dimension adding progressively finer-grained detail.

The name comes from Russian nesting dolls: a 1536-dimension Matryoshka embedding contains a fully functional 768-dimension embedding inside its first half, which contains a functional 384-dimension embedding inside its first half, and so on. You don't need a special decoder or reduction network. You just slice the array.

Compare this to a standard embedding model trained with a single loss on the full vector. Information there is smeared across all dimensions somewhat uniformly, so truncating a normal embedding to 25% of its size destroys a large and unpredictable share of its semantic signal. Truncating a Matryoshka embedding to 25% degrades gracefully because that's exactly the scenario it was trained for.

Many current-generation embedding APIs (OpenAI's text-embedding-3 family, Google's Gemini embedding models, and several open models on Hugging Face like Nomic's and Snowflake's Arctic Embed series) support this natively, either via an explicit dimensions parameter at call time or by documenting that truncation is supported and validated.

Truncating embeddings correctly

If your embedding provider supports a dimensions parameter, use it at request time rather than truncating client-side. This lets the provider skip computing the unused dimensions server-side in some implementations, and guarantees you get the exact behavior they tested.

from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model="text-embedding-3-large",
    input="Matryoshka embeddings let you truncate without retraining.",
    dimensions=256
)

vector = response.data[0].embedding
print(len(vector))  # 256

If you already have full-size embeddings stored and want to truncate them yourself (for example, to test different sizes against a cached dataset without re-embedding), slice the array and then re-normalize. This step matters and is the most common mistake teams make.

import numpy as np

def truncate_embedding(embedding: list[float], target_dim: int) -> list[float]:
    vec = np.array(embedding[:target_dim], dtype=np.float32)
    norm = np.linalg.norm(vec)
    if norm > 0:
        vec = vec / norm
    return vec.tolist()

full_vector = get_cached_embedding(doc_id)       # length 1536
small_vector = truncate_embedding(full_vector, 256)

Skipping the re-normalization step is the single most common bug here. Cosine similarity divides by the vector norm internally in most math libraries, so it seems harmless to skip. But many vector databases (pgvector, Qdrant, Pinecone) let you pick dot product as the distance metric for performance reasons, and dot product is not scale-invariant. An un-normalized truncated vector will silently return wrong nearest-neighbor rankings under dot product, while looking fine under cosine. Always re-normalize after truncation, and confirm which distance metric your index actually uses.

Matryoshka vs. PCA and other reduction methods

Principal Component Analysis (PCA) is the classic approach: fit a projection matrix on a sample of your embeddings, then project every vector through it to a lower dimension. It works on any embedding model, including ones that were never trained with truncation in mind. But it has real costs:

• You must fit the projection on a representative sample of your data and store that projection matrix as an artifact you version alongside your index.
• Every new document at inference time needs the same projection applied before indexing, and every query embedding needs it applied before search, adding a step and a place to introduce train/inference skew.
• If your data distribution shifts significantly (new document types, new languages), the PCA projection can become stale and you need to refit it.
• Compute cost to fit PCA on a large corpus is nontrivial, and it's an offline batch job you have to schedule and re-run.

Matryoshka truncation avoids all four problems. There's no projection matrix, no fitting step, no artifact to version, and no staleness risk, because the "reduction" is just an array slice baked into how the model was trained. The tradeoff is you're locked into an embedding model that was explicitly trained with MRL. Not every model supports it, and you can't retrofit an existing non-Matryoshka embedding model to gain this property after the fact.

Random projection (the Johnson-Lindenstrauss approach) is faster to apply than PCA and doesn't require fitting on your data, but empirically it preserves less retrieval quality per dimension than either PCA or MRL truncation, so it's rarely the right default in 2026.

A practical decision rule: if your embedding provider's model supports MRL, use dimensionality truncation, not PCA. If you're stuck on a model that doesn't (an older fine-tuned in-house model, for instance), PCA is still a reasonable fallback, just budget for the extra pipeline complexity.

Choosing the right target dimension

There's no universal right answer, but a workable process:

1. Pick 2-3 candidate dimensions to test, typically the model's supported truncation points (256, 512, 768, 1024 are common).
2. Build a small labeled evaluation set: 100-300 query-document pairs from your actual domain, ideally pulled from real user queries or support tickets rather than invented ones.
3. Compute recall@k (does the correct document appear in the top k results) at full dimension and at each candidate dimension.
4. Pick the smallest dimension where recall@k drops less than roughly 1-2 percentage points versus full size. Below that threshold you're usually trading real quality for marginal storage savings.

def recall_at_k(queries, ground_truth, index, k=10):
    hits = 0
    for query_id, query_vec in queries.items():
        results = index.search(query_vec, top_k=k)
        result_ids = {r.id for r in results}
        if ground_truth[query_id] in result_ids:
            hits += 1
    return hits / len(queries)

for dim in [256, 512, 768, 1536]:
    idx = build_index(dim=dim)
    score = recall_at_k(eval_queries[dim], ground_truth, idx)
    print(f"dim={dim}: recall@10={score:.3f}")

In practice, teams frequently find that 256 or 512 dimensions retain the large majority of full-size recall for general-purpose semantic search, while highly technical or legal domains with subtle distinctions between documents sometimes need the full dimension count, or at least a larger truncation point, to hold quality.

Two-stage retrieval: a practical pattern

A pattern worth adopting once you have Matryoshka embeddings available: store both a small and a large truncation, and use them in two stages.

1. Coarse pass: search your full corpus using the small vector (say 256 dimensions) to get a candidate set of a few hundred documents. This pass is fast and can run over a much larger index footprint.
2. Rerank pass: re-score just those few hundred candidates using the larger vector (say 1536 dimensions), which you can afford to load for a small candidate set even if it's too expensive to keep for the whole corpus.

This gets you close to full-dimension retrieval quality on the final result while keeping the bulk of your index at the smaller, cheaper size. Since Matryoshka embeddings are nested by construction, the small vector is literally a prefix of the large one, so you compute and store it once, keep the full vector in cheaper cold storage or a separate rerank-only store, and never run two separate embedding calls.

FAQ

Does Matryoshka truncation work with any vector database? Yes. Truncation happens before the vector reaches your database, so it works with pgvector, Pinecone, Qdrant, Weaviate, or any other store. The database just sees a shorter array. Make sure your index schema and any fixed-dimension constraints are updated to match the new size.

Do I need to retrain or fine-tune anything to use Matryoshka embeddings? No, if you're using a model that was already trained with MRL. You call the same embedding API you already use, either passing a dimensions parameter or truncating the output vector yourself, and re-normalizing.

What happens if I mix full-size and truncated vectors in the same index? Don't. Nearest-neighbor search requires all vectors in a comparison to have the same dimensionality and the same normalization. Pick one target dimension per index and re-embed or re-truncate everything consistently. If you need multiple sizes, use separate indexes, as in the two-stage pattern above.

How much quality do I actually lose by truncating to a quarter of the original size? It depends heavily on the model and your domain, which is why running your own recall@k evaluation matters more than trusting a benchmark number from the model's release notes. As a rough industry pattern, truncating a well-trained Matryoshka model from 1536 to 256 dimensions (a 6x reduction) often costs only a few percentage points of recall on general semantic search tasks, but you should verify this on your own data before shipping it.

Can I apply MRL-style truncation to embeddings from a model that wasn't trained with MRL? Technically you can slice the array, but quality will degrade far more sharply and unpredictably than with a true Matryoshka model, because the model was never trained to concentrate information in the leading dimensions. For non-MRL models, PCA fit on your own data is the more reliable reduction path.

Does truncating embeddings save money on the embedding API call itself, or only on storage? Both, in most cases. If your provider supports the dimensions parameter at request time, you're only paying for what you actually store and search against; the cost of the embedding call itself is typically priced by input tokens processed, not output vector size, so the main savings are in storage, memory, and search compute rather than the API call price.