LangChain Memory Explained: Buffers, Summaries and Vector Stores

Every LangChain application eventually runs into the same wall: your agent forgets what happened five turns ago, or it remembers everything and your token bill triples. Memory in LangChain is not one feature — it is a family of strategies for deciding what past conversation to feed back into the model on the next call. There is no single "correct" memory pattern. There is only a set of tradeoffs between recall quality, token cost, and latency, and picking the right one depends entirely on how long your conversations actually get and what you need to remember. This piece walks through the five patterns you will actually use in production — buffer, window, summary, entity, and vector-store memory — with the cost math and decision criteria that usually get skipped in tutorials.

Why memory is a token-cost problem, not just a UX problem

Before getting into individual patterns, it helps to be explicit about what "memory" is actually doing under the hood. LLMs are stateless. Every call to a chat model is a fresh request — the model has no idea what you said three messages ago unless you re-send it as part of the prompt. Memory in LangChain is really just a strategy for constructing the next prompt. It decides which prior turns, summaries, facts, or retrieved snippets get concatenated into the context window before the current user message.

This framing matters because it turns "which memory should I use" into a very concrete engineering question: how many tokens does this strategy add to every single call, and does that cost buy you anything the user will notice? A memory pattern that adds 200 tokens per turn on a chatbot that runs 10,000 conversations a day is a real line item. A memory pattern that silently drops the one fact your user told you in turn 2 is a real churn risk. You are always balancing these two failure modes against each other, and the five patterns below sit at different points on that spectrum.

Buffer memory

Buffer memory is the simplest possible strategy: keep every raw turn of the conversation and replay all of it back to the model on each call. No compression, no filtering, no judgment calls about what matters. If the user said it, and the assistant said it, it goes back in the prompt next time.

The appeal is obvious. Buffer memory is trivial to implement, trivial to debug (you can literally read the whole history and see exactly what the model saw), and it never loses detail. For a support bot handling a five-message exchange, or a coding assistant working through a short debugging session, buffer memory is the right default. There is nothing to compress and nothing to lose, so why introduce that risk.

The problem is what happens as the conversation grows. Token usage per turn scales linearly with conversation length — turn 50 sends everything from turns 1 through 49 plus the new message. Two consequences follow directly from that. First, cost per turn keeps climbing over the life of the conversation, which means your average cost-per-conversation is dominated by whichever users have the longest sessions. Second, you will eventually hit the model's context window ceiling, at which point buffer memory doesn't degrade gracefully — it just breaks, either by truncating silently (losing the oldest and often most important context, like the user's original request) or by erroring out.

Buffer memory is the right choice when you have good reason to believe conversations stay short — a handful of turns, a bounded task, a session that naturally resets. It is the wrong choice for anything designed to run for a while: long-running agents, multi-day support threads, or assistants meant to feel like they "know" the user over time.

Window memory

Window memory takes the buffer approach and puts a lid on it: instead of keeping the entire history, you keep only the last N turns — say, the most recent 6 or 10 exchanges — and everything older simply falls off the end.

This solves the unbounded-growth problem immediately. Token usage per turn is now capped at a predictable ceiling: roughly N turns' worth of tokens, no matter how long the conversation has been running. That predictability is genuinely valuable for capacity planning — you can compute a hard upper bound on cost per call and design your context budget around it, which you cannot do with plain buffer memory.

The cost of that predictability is context loss, and it is an abrupt, not graceful, kind of loss. Turn 11 has zero visibility into turn 1 if your window is set to 10. If the user mentioned a critical constraint early on — "I'm allergic to shellfish," "our deploy target is ARM, not x86," "I already tried restarting the service" — and the conversation runs past the window, that information is just gone. The model will confidently ignore it, or worse, ask the user to repeat something they already said, which is one of the more visible ways a memory system can erode trust in a product.

Window memory works well when recency is what actually matters — a debugging session where only the last few attempts are relevant, a brainstorming tool where old ideas are meant to be superseded, or any interaction where "what did we just say" matters far more than "what did we establish an hour ago." It is a poor fit whenever early context needs to persist for the life of the conversation — onboarding details, stated preferences, constraints given once and expected to be honored forever after.

Picking N is itself a tuning problem worth taking seriously. Too small and you are effectively back to no memory at all; too large and you have just built an expensive, complicated version of buffer memory with an arbitrary cutoff. A reasonable starting point is to look at your actual conversation logs (if you have them) and find the turn distance beyond which older context is rarely referenced again — that is your N.

Summary memory

Summary memory is where things get more interesting, because it introduces compression instead of truncation. Rather than dropping old turns wholesale, you periodically fold them into a running summary — a compact paraphrase of "what has happened so far" — and it's this summary, not the raw transcript, that gets prepended to the prompt going forward.

The mechanics look like this: you keep a small buffer of the most recent raw turns (for fidelity on what just happened), and separately maintain a summary string. When the raw buffer gets too long, you ask the LLM to fold the oldest turns into the existing summary, producing an updated summary, and then clear those turns out of the raw buffer. The next prompt is built from summary + recent raw turns + new message.

def update_summary(llm, existing_summary, new_turns):
    prompt = f"""
    Current summary of the conversation so far:
    {existing_summary}

    New conversation turns to fold in:
    {new_turns}

    Produce an updated summary that incorporates the new turns.
    Keep it concise. Preserve names, decisions, and stated
    preferences. Drop small talk and resolved side-tangents.
    """
    return llm.invoke(prompt)

# Called whenever the raw turn buffer exceeds a threshold,
# e.g. every 6 turns:
running_summary = update_summary(llm, running_summary, oldest_turns)
raw_buffer = raw_buffer[-recent_n:]  # keep only the newest turns

This is conceptual, not a drop-in implementation — the actual LangChain memory classes wrap this pattern with prompt templates and chat-history objects — but the core loop is exactly this: summarize, replace, repeat.

The tradeoff here is genuinely a quality-versus-detail tradeoff, and it is worth being honest about it rather than pretending summary memory is a free lunch. Token usage per turn stays roughly flat over time, similar to window memory, because the summary itself grows slowly compared to raw transcript — that is the win. But every summarization pass is a lossy compression step performed by an LLM, and LLM summarization is not guaranteed to preserve the detail you actually need later. A summary might faithfully retain "the user wants a budget travel itinerary" while quietly dropping "the user specifically said no overnight flights," because that detail seemed minor to the summarizing model at the time but turns out to matter enormously three turns later.

There's also a compounding-error risk worth naming: if you summarize a summary (summarize the summary of turns 1-10 together with turns 11-20 to produce a new summary), small compression losses can accumulate across many cycles, similar to repeatedly re-compressing a JPEG. In practice this is manageable — keep the summarization prompt explicit about what categories of information must never be dropped (stated constraints, decisions, names, numbers) — but it does mean summary memory needs occasional spot-checking in a way that buffer memory does not, since buffer memory can never lose information it never had to compress.

Summary memory is the right call for long-running conversations where full fidelity to every past sentence is less important than maintaining a coherent sense of "where we are" — long customer support threads, ongoing project-assistant conversations, or agents that need to persist across many sessions without the cost of buffer memory. It's a poor fit when the specific wording of something said earlier matters (legal or medical contexts, for instance, where paraphrase risk is unacceptable) or when conversations are short enough that summarization overhead isn't earning its keep.

Entity and fact memory

Entity memory (sometimes called fact memory) takes a different angle entirely: instead of compressing the conversation as prose, it extracts structured facts about specific entities — people, projects, preferences, configurations — mentioned during the conversation, and persists those facts independently of the conversational flow.

Concretely, this usually means running an extraction pass (often another LLM call, sometimes a simpler NER or rule-based step) that looks at each turn and asks "does this contain a durable fact worth remembering," then stores it keyed by entity — something like {"user": {"timezone": "IST", "role": "backend engineer"}, "project_x": {"deploy_target": "kubernetes", "status": "blocked on auth"}}. On future turns, only the facts relevant to entities mentioned in the current message get pulled back into the prompt, rather than the entire history or even the entire fact store.

The cost profile here is attractive precisely because it's selective. Token usage per turn depends on how many relevant facts exist for the entities in play, not on how long the conversation has run — a conversation that's been going for 200 turns costs the same to prompt as one at turn 5, provided the number of distinct facts about the user or project hasn't ballooned. This is a fundamentally different scaling curve than buffer, window, or even summary memory, all of which scale with conversation length in some form. Entity memory scales with the number of durable facts, which is usually a much slower-growing quantity.

The catch is that entity memory only remembers what it's told to look for, and extraction quality depends heavily on your prompting for the extraction step itself. It's good at "the user's name is Priya" or "the API key rotates every 90 days" — discrete, factual, restateable claims. It's bad at preserving conversational nuance, tone, or the reasoning behind a decision ("we chose Postgres over Mongo because of the reporting team's SQL familiarity" is a fact, but the discussion that led there is not something entity memory tries to keep). It also adds an extra LLM call (the extraction step) on top of your main generation call, which is a latency and cost cost of its own, separate from the prompt-size savings on the main call.

Entity memory earns its complexity when your application is fundamentally about tracking specific things over time — a CRM-style assistant, a project-management copilot, an onboarding agent that accumulates a user profile across sessions. It's overkill for a single-session Q&A bot where nothing needs to persist past the conversation ending.

Vector-store-backed long-term memory

The last pattern breaks from the others in a fundamental way: instead of deciding in advance what to keep (recent turns, a summary, extracted facts), vector-store-backed memory keeps everything — embedded — and decides what to retrieve at query time based on semantic similarity to the current message.

Every past interaction (or chunk of one) gets embedded and stored in a vector database. When a new message comes in, you embed it too, run a similarity search against the store, and pull back the top-k most relevant past snippets — not the most recent ones, the most relevant ones. Those get inserted into the prompt alongside (or instead of) a short buffer of recent turns. This is essentially RAG applied to the conversation's own history rather than to an external knowledge base.

This is the only pattern in this list where token cost per turn is close to constant regardless of total history size, and total history size can be enormous — thousands of turns, months of interaction — without the per-turn prompt growing at all, because you're only ever pulling back a fixed k of relevant snippets. This makes it the only real option once conversations get long enough, or numerous enough, that summary or entity memory alone stop being enough (or when you specifically need to resurface something said a long time ago that a recency-biased window or summary would have already dropped or compressed away).

The tradeoffs are different in kind, not just degree, from the earlier patterns. Retrieval quality depends on embedding quality and how you chunk conversation history — chunk too coarsely and you retrieve noisy, only-partially-relevant blocks; chunk too finely and you lose surrounding context that made a snippet meaningful. There's also latency: a similarity search is an extra round trip before your main generation call, on top of the embedding call for the incoming message. And critically, semantic similarity is not the same thing as relevance — a message about "deployment issues" might retrieve a semantically similar but practically unrelated past conversation about a different deployment, and the model has no built-in way to know the retrieved snippet is stale or from an unrelated context unless you tag and filter for that explicitly (by session, by project, by date).

Vector-store memory is the right tool when conversations are long-lived across sessions (a personal assistant that should remember something from three weeks ago), when the relevant history is sparse relative to total volume (most of what was said doesn't matter for this particular question, but the two turns that do matter could be anywhere), or when you're building something closer to a knowledge base with a chat interface than a single continuous dialogue. It's usually overkill for anything that fits comfortably in a summary or a bounded window — the retrieval infrastructure (embedding pipeline, vector store, similarity search, freshness filtering) is real engineering overhead that only pays for itself once the "needle in a haystack" problem is real.

Picking a pattern based on conversation length

None of these patterns is universally correct, and the honest answer to "which memory should I use" is "it depends on how long your conversations run and what kind of forgetting is acceptable." A rough way to think about it:

• Very short, bounded conversations (a handful of turns, task completes and session ends): buffer memory. There's nothing to compress and no cost problem worth solving.
• Medium-length conversations where only recent context matters: window memory. Cap the cost, accept that old context disappears, and make sure that's actually acceptable for your use case.
• Long-running conversations where a coherent narrative matters more than verbatim recall: summary memory. Budget for occasional detail loss and write your summarization prompt carefully to protect the facts that matter most.
• Applications centered on tracking specific things about a user or project over time: entity memory, often layered on top of one of the above for the conversational flow itself.
• Cross-session, high-volume, or "remember anything from anytime" requirements: vector-store-backed memory, accepting the added infrastructure and retrieval-quality tuning it demands.

In practice, production systems often combine these rather than picking exactly one — a short window of raw recent turns for immediate coherence, a running summary for the medium-term arc of the conversation, and a vector store for long-term recall across sessions, with entity memory layered in wherever the application genuinely needs to track discrete facts about a user or project. The mistake to avoid is reaching for the most sophisticated option by default. Vector-store memory solves a real problem, but it solves it at a real cost in infrastructure and tuning effort, and that cost is wasted if your actual conversations rarely run long enough to need it. Start with the cheapest pattern that matches your expected conversation length, and only add complexity once you have evidence — from real usage, not speculation — that the cheaper pattern is actually losing context your users care about.

If you want to go deeper into designing memory systems that hold up under real production load — including how to combine these patterns, how to tune summarization prompts so they don't quietly drop what matters, and how to build retrieval pipelines that stay fast as history grows — that's exactly what we cover in our "Scaling Memory for AI Agents" course, where we build each of these patterns from scratch and stress-test them against long, messy, real-world conversations rather than toy examples.