Metadata Filtering in RAG: Precise Retrieval at Scale

AUTHOR: Pramod Dutta

RAG metadata filtering is the practice of narrowing a vector search to a subset of your corpus using structured attributes (tenant ID, document type, date, department, permission level) before or during the similarity search, instead of relying on embeddings alone to find the right chunk. If you've shipped a RAG pipeline that worked great in a demo with 200 documents and then started returning irrelevant or leaked content once real users loaded in 50,000 documents, the fix is almost never a better embedding model. It's metadata filtering.

This article covers why pure vector search breaks down at scale, how to design a metadata schema that actually holds up, the mechanics of pre-filtering versus post-filtering across the major vector databases, and the access-control patterns that let you build multi-tenant RAG systems without leaking data between customers.

Why vector similarity alone fails at scale

Cosine similarity and dot product search find chunks that are semantically close to a query. They do not know anything about who the document belongs to, when it was written, whether it's a draft or a published version, or whether the user asking the question is even allowed to see it.

At small scale this doesn't matter much. If your knowledge base has 500 chunks, the top-k semantically similar results are usually also the correct ones, because there's little competing noise. Once you cross a few thousand documents, three failure modes show up reliably:

• Cross-tenant leakage. In a multi-tenant SaaS product, a query from Customer A's employee can retrieve a semantically similar chunk from Customer B's uploaded documents if nothing stops it. This is not a hypothetical: it is the single most common production RAG bug reported by teams building B2B AI products.
• Stale document collisions. A policy document gets revised every quarter, but old versions stay in the vector store. A query like "what is our current refund window" can just as easily retrieve the 2023 policy as the 2026 one, because both are semantically about refund windows.
• Irrelevant-but-similar noise. Product documentation and marketing copy about the same product often use overlapping vocabulary. Without a doc_type filter, a "how do I configure X" query can surface a blog post that mentions configuration in passing instead of the actual configuration guide.

Metadata filtering solves all three by constraining the candidate set before similarity ranking ever gets a vote.

Designing a metadata schema that survives contact with production

The mistake most teams make is treating metadata as an afterthought, bolted on after the embedding pipeline is already built. Design the schema first, alongside your chunking strategy, not after.

A metadata schema for a production RAG system typically needs three categories of fields:

Identity and access fields. These are the fields that answer "who is allowed to see this chunk."

• tenant_id: required in any multi-tenant system, even if you think you'll only ever have one customer
• owner_id or team_id: for per-user or per-team document scoping within a tenant
• access_level: an enum like public, internal, confidential if you need row-level security beyond tenant isolation

Structural fields. These describe what the chunk is, independent of its content.

• doc_type: policy, faq, api_reference, blog_post, support_ticket
• source: the originating system (Confluence, Notion, a PDF upload, a Zendesk export)
• document_id and chunk_index: so you can reconstruct the parent document or fetch neighboring chunks for context expansion

Temporal and versioning fields. These are what save you from stale-document collisions.

• created_at / updated_at: ISO 8601 timestamps, not free text
• is_current: a boolean flag set to false the moment a document is superseded, so you can filter it out without deleting it
• version: an integer or semver string if documents go through formal revisions

A concrete chunk record ends up looking like this before it's embedded and inserted:

{
  "id": "chunk_8f3a1c",
  "text": "Refunds are processed within 5 business days of...",
  "embedding": [0.0123, -0.0456, ...],
  "metadata": {
    "tenant_id": "acme-corp",
    "doc_type": "policy",
    "source": "confluence",
    "document_id": "doc_442",
    "chunk_index": 3,
    "created_at": "2026-03-14T00:00:00Z",
    "is_current": true,
    "version": 4,
    "access_level": "internal"
  }
}

Keep field names flat and consistent across your entire pipeline. Nested metadata objects work in some vector databases but not others, and inconsistency here is the number one cause of "the filter silently returns zero results" bugs. If you support both Pinecone and Qdrant in the same codebase because you're testing a migration, flatten everything.

Pre-filtering vs. post-filtering vs. hybrid filtering

There are three mechanically different ways a vector database can combine a metadata filter with a similarity search, and the difference matters for both correctness and latency.

Post-filtering runs the similarity search first, gets the top-k nearest neighbors, and then discards any that don't match the metadata filter. This is the naive approach and it has a serious flaw: if your filter is selective (say, it matches only 2% of the corpus) and you ask for top-10 results, you can easily end up with zero or one result after filtering, even though 500 matching documents exist somewhere further down the similarity ranking. Some SDKs implement this by default if you don't explicitly ask for filtering to happen at the index level, so check your library's behavior rather than assuming.

Pre-filtering applies the metadata filter first, shrinking the candidate set, and then runs similarity search only within that subset. This gives correct results but can be slow if the underlying index doesn't have a fast way to compute "which vectors match this filter" without a full scan, especially at high dimensionality.

Filtered vector search (sometimes marketed as "metadata-aware indexing") is what modern vector databases implement to get correctness without the pre-filtering performance cliff. The database builds the filter into the traversal of the approximate nearest neighbor graph itself, so it explores only nodes that satisfy the filter as it searches, instead of filtering before or after as a separate pass. This is the approach you want in production, and by 2026 it's the default behavior in most mainstream vector databases rather than an opt-in mode.

Here's the same filtered query expressed across a few common vector databases. The syntax differs but the intent, "search within this subset only," is identical.

Pinecone:

results = index.query(
    vector=query_embedding,
    top_k=10,
    filter={
        "tenant_id": {"$eq": "acme-corp"},
        "doc_type": {"$in": ["policy", "faq"]},
        "is_current": {"$eq": True}
    },
    include_metadata=True
)

Qdrant:

from qdrant_client.models import Filter, FieldCondition, MatchValue

results = client.search(
    collection_name="docs",
    query_vector=query_embedding,
    query_filter=Filter(
        must=[
            FieldCondition(key="tenant_id", match=MatchValue(value="acme-corp")),
            FieldCondition(key="is_current", match=MatchValue(value=True))
        ]
    ),
    limit=10
)

Weaviate:

result = (
    client.query
    .get("Document", ["text", "doc_type"])
    .with_near_vector({"vector": query_embedding})
    .with_where({
        "operator": "And",
        "operands": [
            {"path": ["tenant_id"], "operator": "Equal", "valueText": "acme-corp"},
            {"path": ["is_current"], "operator": "Equal", "valueBoolean": True}
        ]
    })
    .with_limit(10)
    .do()
)

pgvector (Postgres), where filtering is just a normal SQL WHERE clause combined with a vector distance operator:

SELECT id, text, embedding <=> %s AS distance
FROM chunks
WHERE tenant_id = %s
  AND is_current = true
  AND doc_type = ANY(%s)
ORDER BY distance
LIMIT 10;

The pgvector case is worth calling out because it's the simplest mental model: metadata filtering is nothing more exotic than a WHERE clause your database already knows how to optimize with a normal B-tree index, as long as you index the filtered columns.

Indexing metadata for filter performance

A metadata filter is only fast if the underlying storage has an index on the fields you're filtering on. This is easy to forget because vector databases put so much emphasis on the ANN index that the metadata side feels like an afterthought, but it behaves like a database problem because it is one.

Practical guidance:

• Index every field you filter on in WHERE-style predicates, not just the ones you display. A tenant_id filter on an unindexed column in a million-row Postgres table with pgvector will force a sequential scan before the vector operator even runs.
• Prefer low-cardinality categorical fields (doc_type, is_current) for pre-filter-style narrowing, since these are cheap to index and highly selective when combined.
• High-cardinality fields like tenant_id in a multi-tenant system are usually your single most important filter and deserve a dedicated index or even physical partitioning (separate collections/namespaces per tenant) once you're past a few hundred tenants.
• Composite indexes matter when you always filter on the same combination of fields together, e.g. (tenant_id, is_current, doc_type). Check your query patterns before assuming a single-column index is enough.
• Re-check filter selectivity as your data grows. A doc_type = 'faq' filter that narrowed 100,000 chunks down to 500 last year might narrow 5,000,000 chunks down to 25,000 this year, and 25,000 candidates is a very different performance profile for the ANN search step.

Namespace and collection isolation for multi-tenancy

Metadata filtering on a shared index is the right default for most multi-tenant RAG systems, but it is not the only isolation strategy, and picking the wrong one for your scale creates real risk.

Shared index with a `tenant_id` filter is the simplest to operate: one collection, one embedding pipeline, one set of indexes to maintain. The risk is entirely in your application code: if a single query path forgets to attach the tenant filter, that request can return data across tenants. Treat the tenant filter as non-optional at the query-builder level, not as something each call site remembers to add. Wrap your retrieval function so the filter is injected automatically and cannot be bypassed by a caller that forgets it.

def search(query_embedding, tenant_id, extra_filter=None, top_k=10):
    base_filter = {"tenant_id": {"$eq": tenant_id}}
    if extra_filter:
        base_filter.update(extra_filter)
    return index.query(vector=query_embedding, top_k=top_k, filter=base_filter)

Every retrieval call goes through this function. There is no code path where tenant_id is optional. This single design decision prevents the majority of cross-tenant leakage bugs.

Per-tenant namespaces (Pinecone namespaces, Qdrant collections, or separate Postgres schemas) give you physical isolation instead of logical isolation. A query against Tenant A's namespace cannot structurally return Tenant B's data, even if your application code has a bug. This costs more operationally (schema migrations run per-namespace, and very small tenants pay fixed per-collection overhead in some databases) but it's the right call once you have a compliance requirement (SOC 2, HIPAA) that a reviewer will actually check, or once a single tenant's data volume is large enough that a shared index's filter selectivity starts to suffer.

A reasonable rule: use a shared index with a mandatory filter for anywhere from a handful up to a few hundred small-to-medium tenants, and switch to per-tenant namespaces once any single tenant is large enough to dominate the index, or once a contract or regulation requires physically separate storage.

Combining metadata filters with hybrid search and re-ranking

Metadata filtering narrows the candidate pool; it does not replace ranking quality. In practice the strongest retrieval pipelines combine three layers:

1. Metadata filter narrows the corpus to the eligible subset (tenant, permissions, document type, currency).
2. Hybrid search (dense vector similarity plus sparse keyword search like BM25) ranks within that subset, since pure embeddings miss exact matches on product codes, error strings, and proper nouns that keyword search catches reliably.
3. Re-ranking with a cross-encoder re-orders the top 20-50 hybrid results into the final top-5 to 10 that actually go into the LLM's context window, since cross-encoders score query-document pairs more accurately than the bi-encoder embeddings used for the initial search.

The order matters: filter first, since it's cheap and correctness-critical; retrieve broadly within the filtered set (ask for more candidates than you'll ultimately use, e.g. top-40 instead of top-10); then re-rank down to what you'll actually pass to the model. Skipping the "retrieve broadly, then re-rank down" step is a common mistake, since asking for exactly top-10 at the vector search stage and then re-ranking those same 10 items adds latency without adding much accuracy, because the correct answer may not have even been in that initial 10.

Debugging filters that silently return nothing

Metadata filters fail in a specific, frustrating way: they don't error, they just return an empty or near-empty result set, and the LLM confidently hallucinates an answer from whatever thin context it got instead of surfacing the retrieval failure. A short checklist before you assume the embedding model is at fault:

• Type mismatches. A created_at stored as a string in one ingestion path and a Unix timestamp in another will make range filters silently miss half your data. Enforce a single schema at write time, ideally with a validation layer, not just a convention documented somewhere.
• Case sensitivity. doc_type: "FAQ" inserted by one pipeline and doc_type: "faq" filtered by another is a real, common bug. Normalize casing on write.
• Over-narrow filters combined with a small top-k. If you filter to is_current: true AND doc_type: "policy" AND tenant_id: "acme-corp" and that combination matches exactly 3 chunks in the whole corpus, asking for top_k: 10 isn't wrong, it's just going to return 3. That's often correct behavior, not a bug, so check actual matching counts before assuming something's broken.
• Filter syntax differs by database. $in, $eq, Equal, must, ANY() all mean roughly the same thing across Pinecone, Weaviate, Qdrant, and pgvector, but mixing up the syntax for the wrong library is an easy copy-paste mistake when you've worked across more than one vector database, and it usually fails silently (empty results) rather than throwing an error.
• Stale metadata after updates. If you update a document's is_current flag in your primary database but don't re-sync it to the vector store, the vector store's copy is now wrong and filtering on it will produce incorrect results indefinitely, until the next full re-index.

Always log the filter object and the raw match count alongside the query itself in development. An empty-context RAG failure is easy to misdiagnose as a bad embedding or a chunking problem when it's actually a filter that matched zero documents.

FAQ

Does metadata filtering slow down vector search? It depends on the database and the indexing strategy. With filtered ANN indexing (the default in most modern vector databases as of 2026) and properly indexed metadata fields, filtering adds negligible latency and often speeds up search by shrinking the candidate space. Post-filtering after retrieval, or filtering on unindexed fields, is where you'll see real slowdowns.

Can I filter on metadata that isn't in the embedded text? Yes, and this is the whole point. Metadata like tenant_id, access_level, or created_at should never be embedded into the vector itself. Store it as structured metadata alongside the chunk and filter on it directly. Embedding metadata into the text (e.g. prepending "Tenant: acme-corp" to every chunk before embedding) is a workaround some teams use with vector databases that don't support metadata filtering well, but it's fragile and wastes context, so prefer a database with real filter support.

Should I use metadata filtering or a separate vector index per tenant? Use a shared index with a mandatory filter for most small-to-medium multi-tenant setups, since it's simpler to operate and cheaper. Move to per-tenant namespaces or collections once you have a hard compliance requirement for physical data separation, or once individual tenants are large enough that shared-index filter performance degrades.

How many metadata fields is too many? There's no fixed number, but the practical limit is usually about how many fields you actually filter on in real queries, not how many you could imagine filtering on. Every extra field is another thing to keep in sync between your source system and the vector store. Start with the identity, structural, and temporal fields covered above, and add more only when a real query pattern needs them.

Does metadata filtering replace the need for row-level security in the LLM's final response? No. Metadata filtering controls what gets retrieved from the vector store, but you still need to validate at the application layer that the user making the request is authorized to see the retrieved chunks, especially in systems where permissions can change after ingestion (a document gets un-shared, an employee is removed from a project). Treat retrieval-time filtering as the first layer of defense, not the only one.

What happens if I forget to set a metadata field during ingestion? Most vector databases treat a missing field as simply absent, which means a filter like is_current: {"$eq": true} will exclude that chunk entirely rather than matching it, since it has no value to compare. This tends to manifest as documents silently vanishing from retrieval results. Set sensible defaults at ingestion time and validate that required fields are present before writing to the vector store, rather than discovering the gap in production.