LangChain

LangChain for Summarization: Map-Reduce and Refine Strategies

Pramod Dutta · Jun 26, 2026 · 14 min read

Why Summarization Breaks the Naive Approach

Every developer who builds an LLM app eventually hits the same wall: someone uploads a 40-page PDF, a two-hour meeting transcript, or a folder of support tickets, and asks for "a quick summary." The naive move is to paste the whole thing into a prompt and let the model do its thing. That works fine for a page or two. It falls apart the moment your document exceeds the model's context window, or even when it technically fits but the model starts "forgetting" details buried in the middle — a well-documented phenomenon where recall quality drops for information that isn't near the start or end of the context.

This is exactly the problem LangChain's summarization chains were built to solve. Instead of stuffing everything into one call, you split the document into chunks, summarize each chunk, and then combine those partial summaries in a structured way. Two strategies dominate this space: map-reduce and refine. There's also a simpler stuff approach for small inputs, but the real engineering decisions happen between map-reduce and refine.

In this article we'll build both from scratch, look at what's actually happening under the hood, compare their trade-offs on cost, latency, and quality, and cover the practical failure modes you'll hit in production — chunk boundary artifacts, summary drift, token budget blowouts, and how to debug them. If you're building anything that touches long-form text — legal contracts, customer call transcripts, research papers, codebases — this is foundational knowledge, and it's exactly the kind of workflow we dig into hands-on in the LangChain Tutorial 2026 course.

The Three Summarization Strategies at a Glance

Before diving into code, it helps to understand what each strategy actually does mechanically.

Stuff: Concatenate all chunks into a single prompt and summarize in one LLM call. Simple, cheap on API calls, but bounded by context window size and prone to quality degradation on long inputs.
Map-Reduce: Summarize each chunk independently ("map"), then combine those summaries into a final summary, possibly across multiple combination passes ("reduce"). Parallelizable and scalable to huge documents, but loses cross-chunk context during the map phase.
Refine: Summarize the first chunk, then iteratively pass each subsequent chunk plus the running summary back to the LLM, asking it to refine the summary with new information. Preserves narrative flow and cross-chunk context, but is strictly sequential and slower.

Here's a simple way to think about it: map-reduce is like assigning different sections of a report to different analysts who never talk to each other, then having an editor stitch their notes together. Refine is like one analyst reading the report cover to cover, updating their notes page by page. Both get you a summary. They get there very differently, and the differences matter a lot once you're running this in production.

Setting Up the Environment

Let's get the basics out of the way. You'll need langchain, langchain-openai (or your provider of choice), and a text splitter.

pip install langchain langchain-openai langchain-community tiktoken

A minimal setup looks like this:

import os
from langchain_openai import ChatOpenAI
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import TextLoader

os.environ["OPENAI_API_KEY"] = "your-api-key-here"

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

loader = TextLoader("meeting_transcript.txt")
documents = loader.load()

text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=2000,
    chunk_overlap=200,
)
chunks = text_splitter.split_documents(documents)

print(f"Split into {len(chunks)} chunks")

Two things worth noting here. First, chunk_overlap matters more than people give it credit for — without overlap, a sentence that spans a chunk boundary gets sliced in half, and each half loses meaning. A 10-15% overlap relative to chunk size is a reasonable starting point. Second, RecursiveCharacterTextSplitter tries to split on paragraph breaks first, then sentences, then words, so it degrades gracefully instead of cutting mid-word.

Building a Map-Reduce Summarization Chain

The map-reduce chain in modern LangChain is best built explicitly with LangGraph or LCEL rather than the older load_summarize_chain convenience wrapper, because explicit construction gives you control over prompts, batching, and error handling. Here's a hand-rolled version that mirrors what the library does internally.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

map_prompt = ChatPromptTemplate.from_template(
    "Summarize the following section of a document in 3-4 sentences, "
    "focusing on key facts, decisions, and action items:\n\n{text}"
)

reduce_prompt = ChatPromptTemplate.from_template(
    "You are combining several partial summaries of a single document "
    "into one coherent final summary. Remove redundancy, preserve all "
    "distinct facts, and keep it under 300 words.\n\n"
    "Partial summaries:\n{text}"
)

map_chain = map_prompt | llm | StrOutputParser()
reduce_chain = reduce_prompt | llm | StrOutputParser()

def map_reduce_summarize(chunks, batch_size=4):
    # Map step: summarize each chunk independently
    chunk_texts = [c.page_content for c in chunks]
    partial_summaries = map_chain.batch(
        [{"text": t} for t in chunk_texts],
        config={"max_concurrency": 5},
    )

    # Reduce step: combine partial summaries, batching if there are many
    combined = partial_summaries
    while len(combined) > 1:
        batches = [
            combined[i:i + batch_size]
            for i in range(0, len(combined), batch_size)
        ]
        combined = reduce_chain.batch(
            [{"text": "\n\n".join(b)} for b in batches],
            config={"max_concurrency": 5},
        )

    return combined[0]

final_summary = map_reduce_summarize(chunks)
print(final_summary)

Notice the while len(combined) > 1 loop. This is the "reduce" part doing multiple passes — if you have 50 chunks producing 50 partial summaries, and your reduce prompt can only comfortably digest 4-5 summaries at a time without hitting context limits, you need several rounds of reduction before you land on a single final summary. This is sometimes called a "collapse" step, and it's the part people forget when they first implement map-reduce — they assume reduce is always a single call, which breaks the moment the document is large enough.

The batch() method with max_concurrency is doing real work here too. Because each chunk summary is independent, you can fire off requests in parallel instead of looping sequentially, which is where map-reduce earns its speed advantage over refine.

Building a Refine Summarization Chain

Refine takes the opposite approach: it never parallelizes, because each step depends on the output of the previous one.

initial_prompt = ChatPromptTemplate.from_template(
    "Write a concise summary of the following text:\n\n{text}"
)

refine_prompt = ChatPromptTemplate.from_template(
    "Here is an existing summary of a document so far:\n\n{existing_summary}\n\n"
    "Below is additional context from the next section of the document:\n\n{text}\n\n"
    "Refine the existing summary with any new facts, decisions, or details "
    "from the additional context. If the new context doesn't add anything "
    "important, return the existing summary unchanged. Keep it under 300 words."
)

initial_chain = initial_prompt | llm | StrOutputParser()
refine_chain = refine_prompt | llm | StrOutputParser()

def refine_summarize(chunks):
    summary = initial_chain.invoke({"text": chunks[0].page_content})

    for chunk in chunks[1:]:
        summary = refine_chain.invoke({
            "existing_summary": summary,
            "text": chunk.page_content,
        })

    return summary

final_summary = refine_summarize(chunks)
print(final_summary)

This is a plain sequential loop — no batching, no concurrency, because chunk 2's summary literally depends on chunk 1's output. That dependency is both the strength and the weakness of refine. The strength: the model always has the running narrative in front of it, so a fact introduced in chunk 1 and referenced again in chunk 8 gets connected correctly. The weakness: if the model produces a bad summary early on, every subsequent refinement inherits that mistake, and there's no cross-check mechanism to catch it.

Comparing the Trade-offs

The choice between map-reduce and refine isn't cosmetic — it changes your latency, cost, and failure modes.

Latency: Map-reduce parallelizes the map step, so wall-clock time scales closer to O(log n) with proper batching. Refine is strictly sequential, so wall-clock time scales linearly with the number of chunks, and can be painfully slow for documents with 50+ chunks.
Cost: Map-reduce and refine end up using a similar number of tokens overall for the same document, but refine's calls tend to grow slightly larger over time because each call includes the full running summary, whereas map-reduce's map calls are uniformly small.
Coherence: Refine tends to produce a more narratively coherent summary because it's built incrementally with full awareness of prior context. Map-reduce can produce a summary that reads like a list of disconnected facts if the reduce prompt isn't written carefully to enforce narrative flow.
Context loss: Map-reduce loses cross-chunk relationships during the map phase — if a name is introduced in chunk 1 and a critical detail about that name appears in chunk 5, the map step summarizing chunk 5 might not know who that name refers to. Refine doesn't have this problem, since it always sees the accumulated summary.
Failure isolation: If one chunk in map-reduce produces a garbage summary (say, the LLM hedges because the chunk was mostly boilerplate), that garbage is diluted among many other, good summaries in the final reduce step. In refine, a bad summary at step 3 poisons every later refinement, since the model treats the "existing summary" as ground truth and just edits around it.

A practical rule of thumb: use map-reduce for documents where sections are relatively independent — chapters in a report, separate support tickets, distinct news articles. Use refine for documents where narrative order and cumulative context matter — a single meeting transcript, a story, a legal contract where clause 12 modifies clause 4.

Handling Token Budgets and Chunk Sizing

A subtle but important design decision is how you size your chunks relative to your model's context window and your summarization prompt's overhead. If your chunks are too small, you multiply the number of LLM calls unnecessarily and increase the risk of losing context at every boundary. If they're too large, you risk exceeding the context window once you add your prompt template, system instructions, and (for refine) the growing running summary.

A reasonable formula:

def estimate_max_chunk_tokens(model_context_window, prompt_overhead_tokens, output_tokens, safety_margin=0.15):
    """
    Estimate a safe chunk size in tokens given a model's context window.
    """
    usable = model_context_window - prompt_overhead_tokens - output_tokens
    return int(usable * (1 - safety_margin))

# Example: gpt-4o-mini with a 128k context window
max_chunk_tokens = estimate_max_chunk_tokens(
    model_context_window=128_000,
    prompt_overhead_tokens=300,
    output_tokens=500,
)
print(f"Safe chunk size: {max_chunk_tokens} tokens")

For refine chains specifically, remember that the "existing_summary" portion of your prompt grows with each iteration if you don't cap it. Left unchecked over a hundred-chunk document, your running summary itself could balloon and eat into your token budget. The fix is straightforward: instruct the model explicitly to keep the summary under a fixed word count on every refine call (as we did above with "under 300 words"), and actually verify this with a token counter rather than trusting the model to obey the instruction perfectly.

import tiktoken

def count_tokens(text, model="gpt-4o-mini"):
    encoding = tiktoken.encoding_for_model(model)
    return len(encoding.encode(text))

def refine_summarize_with_guard(chunks, max_summary_tokens=400):
    summary = initial_chain.invoke({"text": chunks[0].page_content})

    for chunk in chunks[1:]:
        summary = refine_chain.invoke({
            "existing_summary": summary,
            "text": chunk.page_content,
        })

        token_count = count_tokens(summary)
        if token_count > max_summary_tokens:
            # Force a compression pass if the summary is drifting too long
            compress_prompt = ChatPromptTemplate.from_template(
                "Compress this summary to under {limit} tokens while "
                "preserving all key facts:\n\n{text}"
            )
            compress_chain = compress_prompt | llm | StrOutputParser()
            summary = compress_chain.invoke({
                "text": summary,
                "limit": max_summary_tokens,
            })

    return summary

This kind of guard rail is the difference between a demo that works on a 5-page sample document and a system that survives someone uploading a 300-page annual report.

Debugging Common Failure Modes

A few failure patterns show up again and again once you move from prototype to production.

Boundary hallucination: The model sees a chunk that starts mid-sentence and tries to "fix" the grammar, sometimes inventing a subject or context that wasn't there. Increasing chunk_overlap and, better, splitting on semantic boundaries (paragraphs, sections) rather than raw character counts reduces this significantly.
Redundancy in the reduce step: If your map prompts aren't specific about what to extract, you get partial summaries that overlap heavily, and the reduce step ends up repeating the same fact five different ways. Tightening the map prompt to ask for distinct facts, decisions, and entities (rather than a generic "summarize this") helps the reduce step do less deduplication work.
Summary drift in refine chains: Over many iterations, refine chains can slowly drift away from the original document's tone or emphasis, especially if intermediate chunks are low-information (boilerplate, headers, disclaimers). A cheap mitigation is a pre-filter step that scores each chunk's information density before feeding it into the refine loop, and skips chunks that are mostly noise.
Silent truncation: If a chunk plus your prompt template exceeds the model's context window, some providers truncate silently rather than erroring. Always compute and log token counts per call during development so you catch this before it reaches production.
Inconsistent formatting across map outputs: If your map prompt doesn't enforce a strict output format (bullet points vs. prose, for example), the reduce step gets messy inputs and produces an uneven final summary. Pin the format explicitly in the map prompt.

def debug_chunk_pipeline(chunks, llm_chain):
    """
    Quick diagnostic: log token counts and flag chunks near the limit.
    """
    for i, chunk in enumerate(chunks):
        tokens = count_tokens(chunk.page_content)
        flag = "  <-- near limit" if tokens > 1800 else ""
        print(f"Chunk {i}: {tokens} tokens{flag}")

Running a diagnostic pass like this before your first production run on a new document type will save you an afternoon of confused debugging later.

Choosing Between Map-Reduce, Refine, and Hybrid Approaches

In practice, many production systems don't pick one strategy exclusively — they combine ideas from both. A common hybrid pattern is to run map-reduce for speed, but make the reduce prompt explicitly reconstruction-aware: instruct it to look for cross-references between summaries (names, dates, decisions) and reconcile them, essentially borrowing refine's context-awareness at the reduce stage instead of paying for it at every single map step.

Another hybrid worth knowing: hierarchical map-reduce, where you group chunks by logical section first (using document structure like headers, or a lightweight clustering pass), summarize within each section using refine (since sections tend to be narratively coherent), and then map-reduce across sections (since sections tend to be more independent of each other). This gets you refine's coherence within a section and map-reduce's speed across the whole document.

def hierarchical_summarize(sections):
    """
    sections: list of lists, where each inner list is chunks belonging
    to one logical section of the document.
    """
    section_summaries = []
    for section_chunks in sections:
        section_summaries.append(refine_summarize(section_chunks))

    # Now map-reduce across section summaries
    final = reduce_chain.invoke({"text": "\n\n".join(section_summaries)})
    return final

This pattern scales well for things like multi-chapter reports or long codebases split by module, where each module or chapter has internal coherence but the document as a whole doesn't need cross-chunk narrative tracking everywhere.

Evaluating Summary Quality

Once you have a pipeline running, the next question is whether it's actually good. Don't skip this step — summarization is one of those tasks where output "looks fine" on a glance but silently drops important information. A few practical checks:

Coverage check: Extract key entities (names, dates, numbers, decisions) from the source document with a simple extraction prompt, then verify each appears somewhere in the final summary. Flag any that are missing.
Length ratio: Track your compression ratio (summary length / source length) across runs. A ratio that's wildly inconsistent between similar documents usually signals a prompt or chunking issue.
Human spot-check on a sample: Automate what you can, but periodically read a handful of summaries against their source documents yourself. This catches subtle drift that automated checks miss.
LLM-as-judge for regression testing: When you change a prompt or swap models, use a separate LLM call to score whether the new summary preserves the same key facts as a prior "known good" summary. This isn't a substitute for human review, but it's a cheap early warning system.

eval_prompt = ChatPromptTemplate.from_template(
    "Original document facts:\n{facts}\n\n"
    "Summary to evaluate:\n{summary}\n\n"
    "List any facts from the original that are missing from the summary. "
    "If none are missing, respond with 'COMPLETE'."
)
eval_chain = eval_prompt | llm | StrOutputParser()

def check_coverage(key_facts, summary):
    result = eval_chain.invoke({"facts": key_facts, "summary": summary})
    return result

Wiring this kind of check into a CI-style regression suite for your summarization pipeline is worth the setup time — it's the difference between finding out your prompt change broke coverage on a random Tuesday from a user complaint versus catching it before deploy.

Wrapping Up

Map-reduce and refine solve the same underlying problem — summarizing documents too large for a single LLM call — with fundamentally different trade-offs. Map-reduce buys you speed and parallelism at the cost of cross-chunk context. Refine buys you narrative coherence at the cost of sequential latency and error propagation. Neither is universally "better" — the right choice depends on your document structure, your latency budget, and how much cross-referencing matters for your use case.

The code patterns here — batched map calls, iterative reduce collapsing, sequential refine loops with token guards, and coverage-based evaluation — are the building blocks you'll reuse across almost any long-document LLM pipeline, not just summarization. If you want to go deeper into building these chains with LangGraph, handling streaming outputs, and productionizing them with proper observability and evaluation harnesses, that's exactly what we cover step by step in the LangChain Tutorial 2026 course.