LangChain

LangChain for Multi-Agent Systems: Composing Several Chains

Ira Menon · Jun 29, 2026 · 14 min read

Why a Single Chain Stops Being Enough

Most people meet LangChain through a single chain: a prompt template, a model call, an output parser, done. That pattern works beautifully until the task stops being one task. The moment your product needs to research a topic, then summarize it, then check the summary for factual consistency, then format it for a specific audience, you are no longer writing a chain. You are writing a pipeline of specialists that need to hand work off to each other, sometimes in a fixed order and sometimes based on a decision made mid-flight.

This is where multi-agent systems built on LangChain earn their keep. Instead of one enormous prompt trying to do everything, you break the problem into narrow, well-tested chains, each with a single job, and then compose them into a system where a coordinator decides which chain runs next. The individual chains stay simple and easy to evaluate. The complexity moves into the composition layer, where it belongs, because that is the layer you actually want to reason about, log, and debug.

This article walks through what "multi-agent" actually means in a LangChain context, how to structure chains so they compose cleanly, how routing and shared state work in practice, and where teams commonly get this wrong. We will write real code using LangChain Expression Language (LCEL) and LangGraph, since LangGraph has become the de facto way to orchestrate multi-step, multi-agent flows on top of LangChain's primitives. By the end you should be able to look at a messy, monolithic prompt and see the seams where it wants to split into cooperating agents.

What "Multi-Agent" Really Means Here

The term "agent" gets overloaded. In LangChain's vocabulary, an agent is a component that uses an LLM to decide which action to take next, typically by choosing from a set of tools, rather than following a fixed sequence of steps. A "multi-agent system" is a collection of these decision-making units (or simpler deterministic chains standing in for them) that communicate through a shared state or a message-passing protocol.

There are three composition patterns that cover the vast majority of real systems:

Sequential pipeline: Chain A's output feeds directly into Chain B, which feeds into Chain C. No branching, no revisiting. Good for linear workflows like "extract, then summarize, then translate."
Router / supervisor pattern: A dispatcher chain examines the input (or the current state) and decides which specialist chain should handle it next. This is the backbone of most customer-support and research assistants.
Collaborative loop: Two or more agents pass work back and forth, often with one agent critiquing or verifying another's output, until a stopping condition is met. Think "writer agent" and "critic agent" iterating until the critic approves.

All three patterns share one requirement: a shared, well-typed state object that every chain reads from and writes to. Get that state design right and the rest of the system falls into place. Get it wrong, and you will spend your debugging time chasing which chain silently dropped a field.

Setting Up the Building Blocks

Before composing anything, you need chains that are individually solid. LCEL lets you build a chain by piping a prompt, a model, and a parser together with the | operator. Here is a minimal research-summarizer chain that we will reuse as one node in a larger system.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

summarize_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a precise technical summarizer. Summarize the "
               "input in 3 bullet points. No fluff, no repetition."),
    ("human", "{raw_text}")
])

summarize_chain = summarize_prompt | llm | StrOutputParser()

# A second, independent chain that checks factual consistency
verify_prompt = ChatPromptTemplate.from_messages([
    ("system", "Compare the summary against the source text. Reply with "
               "'CONSISTENT' or 'INCONSISTENT: <reason>'."),
    ("human", "SOURCE:\n{raw_text}\n\nSUMMARY:\n{summary}")
])

verify_chain = verify_prompt | llm | StrOutputParser()

Notice these are two entirely separate, independently testable chains. Neither one knows the other exists. That separation is intentional and it is the single biggest lever you have for keeping a multi-agent system maintainable. If verify_chain starts giving bad results, you can iterate on its prompt in isolation, with unit tests that never touch summarize_chain.

Composing Chains with Shared State

Once you have multiple chains, you need something to own the flow between them. LangGraph models this as a graph where nodes are functions (often wrapping a chain) and edges define the transitions, with a typed state object threaded through every node.

from typing import TypedDict, Optional
from langgraph.graph import StateGraph, END

class ResearchState(TypedDict):
    raw_text: str
    summary: Optional[str]
    verification: Optional[str]
    retries: int

def summarize_node(state: ResearchState) -> ResearchState:
    summary = summarize_chain.invoke({"raw_text": state["raw_text"]})
    return {**state, "summary": summary}

def verify_node(state: ResearchState) -> ResearchState:
    result = verify_chain.invoke({
        "raw_text": state["raw_text"],
        "summary": state["summary"]
    })
    return {**state, "verification": result, "retries": state["retries"] + 1}

def route_after_verify(state: ResearchState) -> str:
    if state["verification"].startswith("CONSISTENT"):
        return "end"
    if state["retries"] >= 2:
        return "end"  # give up gracefully after 2 attempts
    return "retry"

graph = StateGraph(ResearchState)
graph.add_node("summarize", summarize_node)
graph.add_node("verify", verify_node)

graph.set_entry_point("summarize")
graph.add_edge("summarize", "verify")
graph.add_conditional_edges("verify", route_after_verify, {
    "retry": "summarize",
    "end": END
})

app = graph.compile()

result = app.invoke({
    "raw_text": "...(long article text here)...",
    "summary": None,
    "verification": None,
    "retries": 0
})
print(result["summary"])

This small graph already demonstrates the collaborative-loop pattern: the verifier can send work back to the summarizer, and a retry counter prevents infinite loops. Notice that ResearchState is the single source of truth. Every node reads only the keys it needs and writes back a merged dictionary. That discipline — reading narrowly, writing explicitly — is what keeps a ten-node graph debuggable instead of becoming a bag of mutable globals.

The Supervisor / Router Pattern

The pattern most teams reach for first is a supervisor that looks at an incoming request and dispatches it to the right specialist chain. This is common in support bots, coding assistants, and multi-tool research agents.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

router_prompt = ChatPromptTemplate.from_messages([
    ("system",
     "Classify the user request into exactly one category: "
     "'billing', 'technical', or 'general'. Reply with only the "
     "category word."),
    ("human", "{user_message}")
])

router_chain = router_prompt | llm | StrOutputParser()

billing_chain = (
    ChatPromptTemplate.from_messages([
        ("system", "You handle billing questions. Be concise and cite "
                   "specific plan details when relevant."),
        ("human", "{user_message}")
    ]) | llm | StrOutputParser()
)

technical_chain = (
    ChatPromptTemplate.from_messages([
        ("system", "You are a technical support specialist. Ask for "
                   "reproduction steps if the issue is unclear."),
        ("human", "{user_message}")
    ]) | llm | StrOutputParser()
)

general_chain = (
    ChatPromptTemplate.from_messages([
        ("system", "You handle general inquiries in a friendly, brief tone."),
        ("human", "{user_message}")
    ]) | llm | StrOutputParser()
)

def dispatch(user_message: str) -> str:
    category = router_chain.invoke({"user_message": user_message}).strip().lower()
    handlers = {
        "billing": billing_chain,
        "technical": technical_chain,
        "general": general_chain,
    }
    handler = handlers.get(category, general_chain)
    return handler.invoke({"user_message": user_message})

The router itself is just another chain — a classifier — which means you can evaluate it the same way you evaluate any other chain: with a labeled test set and a simple accuracy metric. Do not skip this. A router that misclassifies 15% of incoming requests will quietly degrade the entire system's perceived quality, and because the failure happens at dispatch time, it is easy to miss in spot checks of individual specialist chains.

For anything beyond three or four categories, wire the same dispatch logic into a LangGraph node with conditional edges rather than a plain Python dictionary lookup. The dictionary approach is fine for a demo; the graph approach gives you visibility into state transitions once you add logging or a UI that shows execution traces.

Giving Agents Tools, Not Just Prompts

A chain that only talks to an LLM is limited to whatever the model already knows. Multi-agent systems become genuinely useful when individual agents can call tools — a search API, a database query, a calculator — and incorporate the results before responding. LangChain's tool-calling agents handle the loop of "decide to call a tool, call it, read the result, decide again" for you.

from langchain_core.tools import tool
from langchain.agents import create_tool_calling_agent, AgentExecutor

@tool
def lookup_order_status(order_id: str) -> str:
    """Look up the shipping status for a given order ID."""
    # In production this hits your real order-management system
    fake_db = {"A123": "Shipped, arriving in 2 days", "B456": "Processing"}
    return fake_db.get(order_id, "Order not found")

@tool
def calculate_refund(order_total: float, restocking_fee_pct: float) -> float:
    """Calculate a refund amount after applying a restocking fee percentage."""
    return round(order_total * (1 - restocking_fee_pct / 100), 2)

tools = [lookup_order_status, calculate_refund]

agent_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are an order-support agent. Use tools to look up real "
               "data before answering. Never guess an order status."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}")
])

agent = create_tool_calling_agent(llm, tools, agent_prompt)
order_agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

response = order_agent_executor.invoke({
    "input": "What's the status of order A123, and if I cancel it, "
             "what's my refund on a $200 order with a 10% restocking fee?"
})
print(response["output"])

This order_agent_executor can now be dropped into the supervisor pattern above as one more specialist chain, sitting alongside billing_chain and technical_chain. From the router's point of view, it does not matter that this particular specialist happens to call tools internally — it still just takes a message and returns a string. That interface consistency is what lets you mix simple LCEL chains and full tool-calling agents inside the same multi-agent graph without special-casing anything.

Handling Shared Memory Across Agents

A subtlety that trips up a lot of first attempts at multi-agent systems: each specialist chain, by default, has no memory of what happened before it was called. If your router hands a conversation off to technical_chain and then, three turns later, hands it to billing_chain, the billing chain has no idea the user already described their technical issue — unless you explicitly thread that context through the state.

The fix is to keep conversation history in the shared state object and pass the relevant slice into whichever chain is currently active, rather than relying on each chain to have its own persistent memory.

from langchain_core.messages import HumanMessage, AIMessage

class ConversationState(TypedDict):
    messages: list
    active_agent: str

def format_history_for_prompt(messages: list, max_turns: int = 6) -> str:
    recent = messages[-max_turns:]
    lines = []
    for m in recent:
        role = "User" if isinstance(m, HumanMessage) else "Assistant"
        lines.append(f"{role}: {m.content}")
    return "\n".join(lines)

def technical_node(state: ConversationState) -> ConversationState:
    history = format_history_for_prompt(state["messages"])
    result = technical_chain.invoke({
        "user_message": f"Conversation so far:\n{history}\n\n"
                         f"Latest message: {state['messages'][-1].content}"
    })
    state["messages"].append(AIMessage(content=result))
    return state

This is a deliberately simple approach — a sliding window of recent turns, formatted as plain text and injected into the prompt. It scales fine for most support and assistant use cases. If your conversations run long enough that a sliding window starts dropping important context, that is the point to look at summarization-based memory, where an additional chain periodically compresses older turns into a running summary that gets prepended instead of the raw messages. That summarizer is, itself, just another chain in your system — the pattern is recursive, which is part of why it scales so well conceptually even as implementations grow.

Error Handling and Fallbacks Between Agents

Production multi-agent systems fail in more interesting ways than single chains do, because a failure in one node can cascade into the next. A model timeout in your summarizer should not crash the whole graph; it should trigger a fallback path. LangChain's with_fallbacks and LangGraph's conditional edges both support this, but you get the most control by combining them: use with_fallbacks for chain-level failures (API errors, timeouts) and conditional edges for content-level failures (bad classification, failed verification).

from langchain_openai import ChatOpenAI

primary_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0, timeout=10)
backup_llm = ChatOpenAI(model="gpt-4o-mini", temperature=0, timeout=30, max_retries=1)

resilient_summarize_chain = (
    summarize_prompt | primary_llm.with_fallbacks([backup_llm]) | StrOutputParser()
)

def summarize_node_safe(state: ResearchState) -> ResearchState:
    try:
        summary = resilient_summarize_chain.invoke({"raw_text": state["raw_text"]})
    except Exception as exc:
        summary = f"[summary unavailable: {exc}]"
    return {**state, "summary": summary}

Wrap every node that touches an external call — LLM, tool, database — this way. It costs a few extra lines per node and saves you from an entire graph failing because one specialist timed out. It also gives you a place to log structured error data, which matters enormously once you have five or six agents running and need to figure out which one caused a bad final answer.

Observability: Knowing Which Agent Did What

The hardest part of debugging a multi-agent system is not writing the chains — it is figuring out, after the fact, which agent made which decision and why the final output looks the way it does. A router misclassification three steps upstream can produce an answer that looks like a summarizer bug if you are only looking at the final output.

Two practices help enormously:

Tag every chain invocation with metadata. LangChain's .invoke() and .with_config() accept a tags and metadata argument. Tag each node with its name ("node:summarize", "node:verify") so your tracing tool (LangSmith or your own logging) can group calls by agent.
Persist the full state object after every node, not just the final result. When something looks wrong, you want to see the state as it existed after each hop, not just the input and the final output. This is cheap to do — dump the state dict to a log line or a debug table — and it turns "why did this happen" from a guessing game into a five-minute read.

def logged_node(name, fn):
    def wrapper(state):
        new_state = fn(state)
        print(f"[{name}] state after: {new_state}")
        return new_state
    return wrapper

graph.add_node("summarize", logged_node("summarize", summarize_node))
graph.add_node("verify", logged_node("verify", verify_node))

This wrapper pattern costs almost nothing and pays for itself the first time a user reports "the bot gave a weird answer" and you need to reconstruct exactly what happened without asking them to reproduce it.

Common Mistakes When Composing Chains

A few failure patterns show up again and again in teams building their first multi-agent LangChain system:

One giant prompt pretending to be multiple agents. If you find yourself writing "First act as a researcher, then act as a critic, then act as an editor" inside a single prompt, you have not actually built a multi-agent system — you have built a single fragile prompt that is hard to test and easy to break with any edit. Split it into real chains.
No shared state contract. If different chains expect different shapes of input (one wants a raw string, another wants a dict with three keys), every integration becomes bespoke glue code. Define the state schema first, as a TypedDict or Pydantic model, before writing the chains that will populate it.
Unbounded loops in collaborative patterns. The writer/critic loop is powerful but needs an explicit exit condition — a retry counter, a quality threshold, or a maximum iteration count — or it will occasionally spin until it hits a token or cost limit.
Treating the router as an afterthought. The router is often the highest-leverage, least-tested part of the system. Give it its own evaluation set and revisit its prompt whenever you add a new specialist chain, since new categories change the classification boundary for existing ones too.
No fallback for tool failures. Tools call external systems, and external systems fail. An agent that has no plan for "the API returned a 500" will either crash or, worse, hallucinate a plausible-sounding but wrong tool result.

Avoiding these five mistakes will put your system ahead of a large share of the multi-agent projects that stall out in production.

Where to Go From Here

Composing several chains into a multi-agent system is less about clever prompting and more about disciplined engineering: clear interfaces between chains, an explicit shared-state schema, routing logic that is tested like any other classifier, and error handling that treats every external call as something that can fail. Once those foundations are in place, adding a new specialist agent to an existing graph becomes a routine change instead of a redesign.

The chains and graphs in this article are deliberately minimal so you can see the underlying structure clearly, but the same patterns — sequential pipelines, supervisor routing, collaborative loops, tool-calling specialists, shared memory, and fallback handling — scale directly to production systems with a dozen or more cooperating agents. If you want to go deeper on building and deploying these systems end to end, including LangGraph state machines, persistent memory backends, and real deployment patterns, our LangChain Tutorial 2026 course on teachyou.ai walks through building a complete multi-agent application from scratch, with the same composition discipline covered here applied to a full-scale project.