LangChain Retrievers Explained: From Vector Stores to Ensembles

Why your RAG pipeline is only as good as its retriever

Most people building a Retrieval-Augmented Generation (RAG) pipeline spend their first week obsessing over the language model and their first month realizing the model was never the problem. The model can only answer questions using the context it's handed, and that context is fetched by a retriever. If the retriever pulls the wrong chunks, misses a relevant paragraph, or drowns a good match in ten mediocre ones, the model will confidently produce a wrong answer with a straight face.

This is where LangChain's retriever abstraction earns its keep. A retriever in LangChain is not a single algorithm — it's an interface. Anything that takes a query string and returns a list of relevant documents can be a retriever, whether it's a similarity search over a vector store, a keyword-based search, a call to a web API, or a combination of several strategies voting on the best results. Once you understand the interface, you can swap implementations without touching the rest of your chain, which is exactly the kind of flexibility you want when you're iterating on retrieval quality (and you will iterate — a lot).

In this article we'll go from the simplest possible retriever — a vector store wrapped in .as_retriever() — through the more advanced patterns LangChain ships out of the box: multi-query retrieval, contextual compression, parent-document retrieval, and ensemble retrieval that blends dense and sparse search. By the end, you should be able to look at a retrieval problem and know which tool actually fits, instead of reaching for whatever tutorial you saw last.

The Retriever interface: simpler than it looks

Every retriever in LangChain implements the same contract: a get_relevant_documents method (or its async twin, aget_relevant_documents) that accepts a query string and returns a list of Document objects. That's the entire surface area. Because the interface is so small, retrievers compose easily — you can wrap one retriever inside another, chain them, or swap one out for testing without breaking anything downstream.

Here's the shape of a custom retriever, stripped down to show what's actually required:

from langchain_core.retrievers import BaseRetriever
from langchain_core.documents import Document
from typing import List

class KeywordRetriever(BaseRetriever):
    """A toy retriever that returns documents containing a keyword."""

    documents: List[Document]

    def _get_relevant_documents(self, query: str) -> List[Document]:
        keyword = query.lower()
        return [
            doc for doc in self.documents
            if keyword in doc.page_content.lower()
        ]

docs = [
    Document(page_content="LangChain retrievers wrap vector stores."),
    Document(page_content="Pandas is used for data manipulation."),
]

retriever = KeywordRetriever(documents=docs)
results = retriever.invoke("retrievers")
print(results)

Notice there's no mention of embeddings, cosine similarity, or FAISS anywhere in that code. That's the point — retrieval is a contract, not a technology. Once you internalize that, the rest of this article is just a tour of different implementations of the same contract, each suited to a different failure mode you'll run into with real data.

The default: vector store retrievers

The retriever you'll use 80% of the time is the one you get for free from any vector store. LangChain's vector store classes — Chroma, FAISS, Pinecone, Qdrant, and dozens more — all expose an .as_retriever() method that turns the store into a BaseRetriever with a single line.

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_text_splitters import RecursiveCharacterTextSplitter

# Assume `raw_text` holds your source document as a string
splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100)
chunks = splitter.create_documents([raw_text])

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vectorstore = FAISS.from_documents(chunks, embeddings)

retriever = vectorstore.as_retriever(
    search_type="similarity",
    search_kwargs={"k": 4}
)

relevant_docs = retriever.invoke("How does chunk overlap affect retrieval quality?")
for doc in relevant_docs:
    print(doc.page_content[:120], "\n---")

There are a few knobs worth knowing here. The search_type parameter isn't limited to plain similarity search — you can pass "mmr" (Maximal Marginal Relevance) to reduce redundancy among the top results, which matters a lot when your chunks overlap heavily and the naive top-k search returns four near-duplicate passages instead of four diverse ones. You can also pass "similarity_score_threshold" combined with a score_threshold in search_kwargs to filter out weak matches entirely rather than always returning exactly k documents regardless of relevance.

• similarity — plain nearest-neighbor search by embedding distance, fastest and simplest
• mmr — balances relevance against diversity so you don't get four versions of the same paragraph
• similarity_score_threshold — only returns documents above a minimum relevance score, useful when you'd rather return zero results than a bad one

The mistake most people make here is treating k as a tuning knob for accuracy when it's really a tuning knob for context window budget. Bumping k from 4 to 10 doesn't make retrieval smarter, it just gives the model more chances to find something useful buried in more noise. If your baseline retriever isn't finding the right chunk in the top 4, raising k is a band-aid — the real fix is usually better chunking, better embeddings, or one of the techniques below.

Multi-query retrieval: asking the question five different ways

A single embedded query is a single point in vector space, and it can miss documents that are relevant but phrased differently. If a user asks "why is my API returning 429 errors," the most relevant chunk in your docs might describe "rate limiting policies" without ever using the phrase "429." Cosine similarity on the raw query embedding might not bridge that gap.

MultiQueryRetriever solves this by using an LLM to generate several reformulations of the original query, running similarity search for each one, and merging the deduplicated results.

from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

multi_query_retriever = MultiQueryRetriever.from_llm(
    retriever=vectorstore.as_retriever(search_kwargs={"k": 4}),
    llm=llm,
)

docs = multi_query_retriever.invoke(
    "why is my API returning 429 errors"
)
print(f"Retrieved {len(docs)} unique documents across all query variants")

Under the hood, this generates something like three to five alternate phrasings — "what causes HTTP 429 responses," "how does rate limiting work in this API," "troubleshooting too many requests errors" — searches with each, and unions the results. You trade one extra LLM call (and the latency that comes with it) for meaningfully better recall on queries where the user's vocabulary doesn't match the source documents' vocabulary. This is a good default upgrade for any RAG system fielding questions from real users instead of from people who wrote the documentation.

If latency is a hard constraint, you can cap the number of generated queries or cache the reformulations for common question patterns, but in most support-bot and internal-knowledge-base use cases the extra round trip is worth it.

Contextual compression: filtering out the noise after retrieval

Vector search returns whole chunks, and chunks are rarely a perfect match for the query — a 500-token chunk might contain one relevant sentence and four paragraphs of surrounding context that just eats up your prompt budget and dilutes the signal you're feeding the model. ContextualCompressionRetriever wraps a base retriever and post-processes its output, either by extracting only the relevant snippets from each document or by filtering out documents that turn out not to be relevant on closer inspection.

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

compressor = LLMChainExtractor.from_llm(llm)

compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor,
    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 6}),
)

compressed_docs = compression_retriever.invoke(
    "what is the refund policy for annual subscriptions"
)
for doc in compressed_docs:
    print(doc.page_content)

LLMChainExtractor asks an LLM to pull out only the parts of each retrieved document relevant to the query, discarding the rest. There's also LLMChainFilter, which is cheaper — instead of rewriting document content, it just makes a binary keep-or-discard decision per document, which is useful when your chunks are already reasonably tight and you mainly want to prune false positives rather than trim their contents.

• LLMChainExtractor — rewrites each document down to the relevant snippet, best when chunks are large and only partially relevant
• LLMChainFilter — keeps or discards whole documents, cheaper and faster, best when chunks are already small
• EmbeddingsFilter — filters by embedding similarity threshold instead of an LLM call, the cheapest option of the three

That last one, EmbeddingsFilter, is worth calling out separately because it doesn't cost you an extra LLM call at all — it just re-scores retrieved documents against the query embedding and drops anything below a threshold. It's a good middle ground when you want some noise reduction without doubling your latency and API spend.

Parent-document retrieval: small chunks for search, big chunks for context

Here's a tension every RAG builder runs into: small chunks embed and match more precisely, because a 200-token chunk is topically focused and its embedding isn't diluted by unrelated content. But small chunks make for lousy context — the model needs surrounding paragraphs to actually answer well, not an isolated sentence stripped of everything around it.

ParentDocumentRetriever resolves this by splitting documents into small child chunks for indexing and search, while keeping a mapping back to larger parent chunks (or the full original document) that get returned once a child chunk matches.

from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
from langchain_community.vectorstores import Chroma
from langchain_text_splitters import RecursiveCharacterTextSplitter

parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000)
child_splitter = RecursiveCharacterTextSplitter(chunk_size=300)

vectorstore = Chroma(
    collection_name="parent_child_demo",
    embedding_function=OpenAIEmbeddings(model="text-embedding-3-small"),
)
docstore = InMemoryStore()

retriever = ParentDocumentRetriever(
    vectorstore=vectorstore,
    docstore=docstore,
    child_splitter=child_splitter,
    parent_splitter=parent_splitter,
)

retriever.add_documents(chunks)

results = retriever.invoke("explain the pricing tiers")
print(results[0].page_content)

The search itself happens over the small, precise child chunks, so the similarity match is sharp. But what gets returned to your prompt is the larger parent chunk, giving the model the full surrounding context it needs to actually reason about the answer rather than working from an isolated fragment. This pattern is especially valuable for long-form technical documentation, legal text, or anything where meaning depends heavily on surrounding paragraphs — a single sentence about "the fee" means nothing without the paragraph establishing which fee it's talking about.

Ensemble retrieval: combining dense and sparse search

Pure vector similarity search is excellent at understanding semantic meaning but surprisingly bad at exact-match cases — product codes, error messages, acronyms, or proper nouns that a dense embedding model may not represent distinctly. Sparse retrieval methods like BM25, which score documents by keyword overlap and term frequency, are the opposite: excellent at exact matches, poor at understanding that "car" and "automobile" mean the same thing.

EnsembleRetriever runs multiple retrievers in parallel and merges their ranked results using Reciprocal Rank Fusion, giving you the best of both approaches without having to pick one.

from langchain.retrievers import EnsembleRetriever, BM25Retriever

bm25_retriever = BM25Retriever.from_documents(chunks)
bm25_retriever.k = 4

vector_retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

ensemble_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, vector_retriever],
    weights=[0.4, 0.6],
)

results = ensemble_retriever.invoke("SKU-4471 warranty terms")
for doc in results:
    print(doc.page_content[:100])

The weights parameter lets you tune how much influence each retriever has on the final ranking. A product-support knowledge base full of SKUs, error codes, and part numbers might lean more heavily on BM25 (a higher weight there), while a conceptual FAQ about company policies might lean almost entirely on the vector retriever. In practice, teams that ship this to production usually land somewhere around an even split and then adjust based on a small evaluation set of real queries rather than guessing.

This is also the pattern to reach for the moment you notice your retriever failing on queries containing IDs, codes, or exact terminology — that's the textbook symptom of a pure-dense-retrieval system, and it's one of the cheapest fixes available because BM25 requires no embeddings, no vector database, and almost no extra infrastructure.

Self-querying retrievers: letting the LLM write the filter

Sometimes the query itself contains structured constraints buried in natural language — "show me reviews from 2024 with a rating above 4" is really a semantic search for "reviews" combined with a metadata filter on year and rating. A plain similarity search treats the whole sentence as one embedding and loses the structured part entirely.

SelfQueryRetriever uses an LLM to parse the natural-language query into a semantic search string plus a structured metadata filter, provided you describe your metadata schema up front.

from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain.chains.query_constructor.schema import AttributeInfo

metadata_field_info = [
    AttributeInfo(name="year", description="The year the review was written", type="integer"),
    AttributeInfo(name="rating", description="Star rating from 1 to 5", type="integer"),
]

self_query_retriever = SelfQueryRetriever.from_llm(
    llm=llm,
    vectorstore=vectorstore,
    document_contents="Customer product reviews",
    metadata_field_info=metadata_field_info,
)

results = self_query_retriever.invoke(
    "show me reviews from 2024 with a rating above 4"
)

This only works if your documents actually carry the metadata fields you describe — the retriever can't filter on a rating field that was never attached to the documents during ingestion. That means self-querying is as much an ingestion-pipeline decision as it is a retrieval decision; you need to plan your metadata schema before you need this retriever, not after.

Putting it together: a retrieval strategy, not a single retriever

The biggest shift in thinking that experienced RAG builders go through is realizing that "which retriever should I use" is often the wrong question. The right question is "what retrieval strategy fits my data and my failure modes," and the answer is frequently a composition of two or three of the patterns above rather than any single one in isolation.

A reasonable default for a lot of production systems looks like this:

1. Chunk documents with ParentDocumentRetriever so small chunks drive search precision while large chunks drive answer quality
2. Wrap the vector half in an EnsembleRetriever alongside BM25 so exact-match queries (codes, names, IDs) don't silently fail
3. Add ContextualCompressionRetriever on top when your chunks are large enough that irrelevant content is diluting the prompt
4. Reach for MultiQueryRetriever specifically for user-facing chat interfaces where question phrasing varies wildly from your source vocabulary

You don't need all four on day one. Start with a plain vector store retriever, and only add complexity when you can point to a specific class of query it's failing on. Retrieval debugging is empirical — the fix for "the model gave a wrong answer" is almost never "add more retrievers," it's "look at what documents were actually retrieved for that query and figure out why the right one wasn't in the list." LangChain's invoke() interface makes that inspection trivial since every retriever, no matter how composed, returns the same list of Document objects you can print and eyeball.

Common mistakes that quietly sink retrieval quality

A few patterns show up again and again in RAG systems that underperform:

• Chunking without testing chunk boundaries — splitting mid-sentence or mid-table destroys the semantic coherence a chunk needs for its embedding to be meaningful
• Using the same chunk size for search and for context — as covered above, this is exactly what parent-document retrieval fixes
• Ignoring exact-match queries — if your evaluation set is all conceptual questions, you'll never notice that codes and IDs are failing until a user hits it in production
• Never re-ranking — top-k similarity search alone often surfaces documents that are topically close but not the best answer; a re-ranking step (even a cheap LLMChainFilter pass) catches this
• Treating `k` as a quality lever — as discussed, it's a budget lever, not an accuracy lever

None of these mistakes are exotic. They're the ordinary result of shipping the first retriever that technically works and never revisiting it once the demo looks good. The tutorials make retrieval look like a solved problem because the demo documents are small and clean. Production documents are messy, inconsistent, and full of exactly the edge cases — acronyms, tables, mixed phrasing — that expose a naive retriever's weaknesses.

Where to go from here

Retrievers are the part of a RAG system that determines whether the rest of the pipeline is worth building. A brilliant prompt template and a state-of-the-art model can't compensate for a retriever that hands them the wrong three paragraphs. The good news is that LangChain gives you a full toolbox — vector store retrievers for the common case, multi-query for vocabulary mismatch, contextual compression for noisy chunks, parent-document retrieval for the small-chunk-versus-context tension, ensemble retrieval for exact-match blind spots, and self-querying for structured filters hiding inside natural language.

The skill that actually matters isn't memorizing which class to import — it's learning to diagnose which failure mode you're looking at from a bad retrieval result, and picking the pattern that addresses that specific failure rather than throwing every technique at the problem at once. That diagnostic instinct is exactly what we drill into hands-on projects inside the LangChain Tutorial 2026 course, where you'll build and evaluate each of these retriever patterns against real, messy datasets instead of toy examples — so when your own RAG system starts giving weird answers, you'll already know where to look.