LangChain

LangChain Router Chains: Directing Queries to the Right Handler

Pramod Dutta · Jun 28, 2026 · 15 min read

Why a Single Prompt Stops Working

Every AI application starts the same way: one prompt, one chain, one model call. It works fine in a demo. Then the feature list grows. Users start asking billing questions, technical support questions, product recommendations, and general chit-chat through the same input box, and the one prompt that used to handle everything now handles nothing well. You either bloat the prompt into an unmaintainable wall of instructions, or you accept mediocre answers because the model is trying to be a generalist when the task actually needed a specialist.

This is the exact problem router chains solve. Instead of forcing one prompt to be an expert at everything, you classify the incoming query first, then hand it off to a chain built specifically for that category. A billing question goes to a chain with billing context and a narrow, focused prompt. A technical question goes to a chain that knows your product's architecture. A vague or unrelated question falls through to a default chain that handles the general case gracefully.

Routing is not a new idea in software engineering — it's the same principle behind URL routers in web frameworks and dispatch tables in compilers. What's different in an LLM application is that the "route decision" itself can be made by a model, by a rules-based classifier, or by a mix of both. LangChain gives you the primitives to build this cleanly using the LangChain Expression Language (LCEL), specifically through RunnableBranch and RunnableLambda. This article walks through how routing actually works today, why the old MultiPromptChain approach is on its way out, and how to build a production-shaped router from scratch.

What a Router Chain Actually Does

A router chain has exactly two responsibilities, and keeping them separate is the key to understanding the whole pattern:

Classify — look at the incoming input and decide which category or "route" it belongs to.
Dispatch — send the input to the chain, prompt, or tool associated with that category, and return its output.

That's it. The router itself does not answer the question. It's a traffic cop, not a responder. This separation matters because it lets you reason about each piece independently. You can test your classifier in isolation by feeding it a hundred sample queries and checking the labels it produces. You can test each destination chain independently by calling it directly with a known input. And you can swap the classification strategy — say, from a keyword match to an LLM call to a fine-tuned intent classifier — without touching any of the destination chains at all.

There are broadly three ways to implement the classification step:

Rule-based routing — regex or keyword matching against the input. Fast, free, deterministic, but brittle against paraphrasing.
Embedding-based routing — embed the query and compare it against reference embeddings for each category using cosine similarity. Good middle ground: cheap, no LLM call needed, but requires you to maintain reference examples.
LLM-based routing — ask a model to classify the query into one of your predefined categories, usually with structured output. Most flexible and most robust to phrasing variance, but adds latency and cost.

Most production systems use a hybrid: fast rule-based checks for obvious cases (an exact match on "refund" routes straight to billing) and an LLM classification step as the fallback for anything ambiguous.

Think about what each layer actually buys you before defaulting to the most expensive option. A regex check for "refund," "invoice," or "cancel subscription" costs nothing and runs in microseconds — if that catches thirty percent of your traffic with high precision, you've already cut your classification-LLM bill by nearly a third before the model ever gets involved. Embedding-based routing sits in the middle: you precompute embeddings for a handful of representative examples per category, embed the incoming query, and pick the category with the highest cosine similarity. It handles paraphrasing far better than keyword matching and doesn't need a model call at inference time beyond the embedding itself, which is usually cheaper and faster than a full chat completion. LLM-based classification is the most flexible and the most expensive, and it's the one most tutorials reach for first because it's the easiest to write — but it should usually be the layer you fall back to, not the layer you start with, once you understand your traffic patterns well enough to build the cheaper checks in front of it.

The Old Way: MultiPromptChain and LLMRouterChain

If you've read older LangChain tutorials, you've probably seen MultiPromptChain paired with LLMRouterChain. The pattern looked like this: you defined a dictionary of destination chains, wrote a router prompt template that asked an LLM to output the destination name plus a possibly-rephrased version of the input, parsed that output with RouterOutputParser, and fed the result into MultiPromptChain.from_prompts(...).

It worked, but it had real problems. The router prompt was rigid and hard to customize. The output parser expected a specific JSON-like format that smaller or less-instruction-tuned models would frequently botch, causing parsing failures in production. Debugging what actually happened inside the chain meant digging through several layers of abstraction. And because it predates LCEL, it doesn't compose cleanly with streaming, batching, or async execution the way modern Runnables do.

LangChain's own documentation now steers new work toward LCEL-based routing with RunnableBranch or a custom RunnableLambda, precisely because these give you full control over the classification prompt, the parsing logic, and the dispatch logic, while staying inside the same composable Runnable interface as everything else in your pipeline. This is not a deprecated-and-forgotten API — it still runs — but if you're starting something new, build it on Runnables, not on MultiPromptChain.

RunnableBranch: The Core Primitive

RunnableBranch is LangChain's built-in conditional Runnable. You give it a list of (condition, runnable) pairs plus a default runnable. At invocation time, it walks the conditions in order and executes the first branch whose condition returns True. If none match, it falls back to the default.

Here's a complete, runnable example that routes customer support queries to billing, technical, or general-purpose handlers:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableBranch, RunnableLambda
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

# --- Destination chains ---

billing_prompt = ChatPromptTemplate.from_template(
    """You are a billing support specialist for a SaaS product.
Answer the customer's question about invoices, charges, refunds,
or subscription plans. Be precise about numbers and policies.

Customer question: {question}"""
)
billing_chain = billing_prompt | llm | StrOutputParser()

technical_prompt = ChatPromptTemplate.from_template(
    """You are a senior technical support engineer.
Answer the customer's question about API errors, integration issues,
or bugs. Include a concrete next step or code fix when relevant.

Customer question: {question}"""
)
technical_chain = technical_prompt | llm | StrOutputParser()

general_prompt = ChatPromptTemplate.from_template(
    """You are a friendly product assistant. Answer the customer's
general question as helpfully as possible.

Customer question: {question}"""
)
general_chain = general_prompt | llm | StrOutputParser()

# --- Classifier ---

classifier_prompt = ChatPromptTemplate.from_template(
    """Classify the customer question into exactly one category:
"billing", "technical", or "general". Respond with only the
category word, nothing else.

Question: {question}
Category:"""
)
classifier_chain = classifier_prompt | llm | StrOutputParser()


def classify(input_dict: dict) -> dict:
    category = classifier_chain.invoke({"question": input_dict["question"]})
    return {"question": input_dict["question"], "category": category.strip().lower()}


# --- Router ---

router = RunnableBranch(
    (lambda x: "billing" in x["category"], billing_chain),
    (lambda x: "technical" in x["category"], technical_chain),
    general_chain,  # default branch
)

full_chain = RunnableLambda(classify) | router

result = full_chain.invoke({"question": "Why was I charged twice this month?"})
print(result)

Notice the shape of this: RunnableLambda(classify) runs first and attaches a category field to the payload. RunnableBranch then reads that field and picks a destination chain. Each condition is a plain Python function that takes the current input and returns a boolean — there's no special DSL to learn. The last positional argument to RunnableBranch (with no condition wrapped around it) is always the default, so make sure it's the last item you pass in.

This pattern generalizes to any number of categories. Add a new destination chain, add a new condition tuple, done. You don't touch the classifier logic or any of the other destination chains.

Building a Router with RunnableLambda Instead

RunnableBranch is convenient, but sometimes you want more control than a linear if-elif-else gives you — for instance, dispatching from a dictionary lookup, handling an unknown category with custom logging, or running two chains in parallel before merging results. In those cases, a plain RunnableLambda that does the dispatch itself is often simpler to read and debug than nested branch conditions.

from langchain_core.runnables import RunnableLambda

destination_chains = {
    "billing": billing_chain,
    "technical": technical_chain,
    "general": general_chain,
}


def route_to_chain(input_dict: dict):
    category = input_dict["category"]
    chain = destination_chains.get(category, general_chain)
    return chain.invoke({"question": input_dict["question"]})


router_lambda = RunnableLambda(route_to_chain)
full_chain_v2 = RunnableLambda(classify) | router_lambda

result = full_chain_v2.invoke({"question": "My API key returns a 401 error"})
print(result)

This version is functionally equivalent to the RunnableBranch example but reads more like ordinary Python, which makes it easier to add logging, metrics, or a fallback chain when the category doesn't match anything you expected. If you're the kind of engineer who prefers explicit dictionaries over chained boolean conditions, this is the pattern to reach for. Both approaches are fully compatible with .stream(), .batch(), and .ainvoke() because everything here is a Runnable — that composability is the entire point of building on LCEL instead of the older chain classes.

Structured Output for More Reliable Classification

Asking a model to "respond with only the category word" works most of the time, but models occasionally add a stray period, a full sentence, or wrap the answer in quotes. A more robust approach uses structured output — Pydantic models with with_structured_output — so the classification step returns a guaranteed, typed object instead of a string you have to defensively parse.

from pydantic import BaseModel, Field
from typing import Literal

class RouteQuery(BaseModel):
    """Classify a customer query into the correct support category."""

    category: Literal["billing", "technical", "general"] = Field(
        description="The most relevant support category for this query"
    )
    confidence: float = Field(
        description="Confidence score between 0 and 1", ge=0.0, le=1.0
    )


structured_llm = llm.with_structured_output(RouteQuery)

classifier_prompt_v2 = ChatPromptTemplate.from_template(
    """Classify the following customer support question.

Question: {question}"""
)

structured_classifier = classifier_prompt_v2 | structured_llm


def classify_structured(input_dict: dict) -> dict:
    route = structured_classifier.invoke({"question": input_dict["question"]})
    return {
        "question": input_dict["question"],
        "category": route.category,
        "confidence": route.confidence,
    }

This version buys you two things a raw string classifier can't give you cleanly: a Literal type that the model is constrained to (no more stray output to sanitize), and a confidence score you can act on. If confidence comes back below some threshold, you can route to a human-in-the-loop queue or the general chain instead of trusting a low-confidence classification blindly. That's a meaningful production safeguard that's nearly impossible to bolt onto free-text classification without fragile string parsing.

Confidence scores are only useful if you actually act on them, which brings us to the piece new teams almost always underinvest in: the default branch. It's usually the first thing that breaks in front of real users. Your classifier will misclassify things. Users will ask questions that don't fit any of your categories. Someone will paste three paragraphs of unrelated text into the support box. The fallback chain is not an afterthought — treat it as a first-class destination.

A well-designed default chain should do at least one of these:

Acknowledge that the system isn't fully sure how to categorize the request, and answer as helpfully as it can anyway.
Ask a clarifying question rather than guessing at an answer, when the input is genuinely ambiguous.
Log the unclassified query somewhere you can review later, so you can spot patterns and add a new route for a category you didn't anticipate.
Escalate to a human agent if your application has that option, rather than forcing the model to bluff an answer to a question it has no context for.

def log_and_answer(input_dict: dict) -> str:
    # In production, replace this with a real logging call —
    # a database write, a metrics counter, or a queue message.
    print(f"[UNROUTED QUERY] {input_dict['question']}")
    return general_chain.invoke({"question": input_dict["question"]})


fallback_chain = RunnableLambda(log_and_answer)

router_with_logging = RunnableBranch(
    (lambda x: x["category"] == "billing", billing_chain),
    (lambda x: x["category"] == "technical", technical_chain),
    fallback_chain,
)

That single print call (swap in real logging in production) is often the difference between a router that silently degrades over months and one your team can actively improve, because you'll actually see which queries are slipping through uncategorized.

Testing Router Chains

Because the classifier and the destination chains are separate pieces, you can and should test them separately. Testing the whole pipeline end-to-end with an LLM call for every test case is slow and non-deterministic; testing the routing logic on its own is fast and repeatable.

import pytest

test_cases = [
    ("Why does my invoice show an extra $10 charge?", "billing"),
    ("I'm getting a 500 error when calling /v1/users", "technical"),
    ("What's the weather like today?", "general"),
    ("Can I get a refund for last month's subscription?", "billing"),
]

@pytest.mark.parametrize("question,expected_category", test_cases)
def test_classification(question, expected_category):
    result = classify({"question": question})
    assert result["category"] == expected_category

Run this against your classifier prompt whenever you tweak the wording, add a new category, or change models. LLM-based classifiers drift in subtle ways when you change the underlying model — a prompt tuned for GPT-4o-mini won't necessarily classify identically on a different model — so a small regression suite like this catches routing failures before your users do. For the destination chains themselves, you can test them independently by asserting on structural properties of the output (does the billing chain's answer mention a specific policy term, does the technical chain's answer include a code block) rather than exact string matches, since LLM outputs vary run to run.

When Routing Is (and Isn't) the Right Tool

Router chains earn their complexity when you have genuinely distinct categories of input that benefit from different system prompts, different tools, or different context windows. Support ticket triage, multi-domain Q&A bots, and agents that need to pick between "search the web," "query the database," or "answer from memory" are all good fits.

Routing is the wrong tool when your categories overlap heavily, when a single well-written prompt with a bit of few-shot guidance already handles the variance fine, or when the cost of an extra classification LLM call outweighs the quality gain you get from specialization. Adding a router because it "feels more sophisticated" is a common mistake — it adds a moving part, a failure mode (misclassification), and latency (an extra model round-trip) that a simpler design might not need. Measure the actual quality difference between a single unified prompt and a routed setup on your own evaluation set before committing to the added complexity.

There's also a latency cost that's easy to overlook until it shows up in production monitoring. A routed pipeline is, at minimum, two sequential model calls where an unrouted pipeline is one — the classification call has to finish before the destination chain even starts, so you're adding the classifier's full round-trip time to every single request, not just the ambiguous ones. If your classifier runs on a smaller, faster model (which it usually should, since classification is a much easier task than generating a full answer) that overhead might be negligible. But if you accidentally point your classifier at the same large model you use for final answers, you can end up doubling your response latency for no quality benefit on the easy, obvious cases that didn't need routing help in the first place. Profile the classifier's contribution to total latency separately from the destination chain's contribution — it's a five-minute check that saves you from shipping a router that feels sluggish for reasons that have nothing to do with the answers it's producing.

It's also worth noting that routing composes with agents rather than competing with them. A LangGraph-based agent making dynamic decisions about which tool to call at each step is doing a more general, iterative version of the same routing idea — RunnableBranch is a good fit when you have a small, fixed number of known categories decided once per request; an agent loop is a better fit when the model needs to make several sequential decisions, potentially re-routing mid-conversation based on tool outputs.

Putting It Together: A Realistic Router

Here's a more complete version that combines structured classification, confidence-based fallback, and logging into one cohesive pipeline you could actually ship:

from langchain_core.runnables import RunnableLambda, RunnableBranch

CONFIDENCE_THRESHOLD = 0.6

def classify_and_prepare(input_dict: dict) -> dict:
    route = structured_classifier.invoke({"question": input_dict["question"]})
    return {
        "question": input_dict["question"],
        "category": route.category,
        "confidence": route.confidence,
    }

def is_confident_category(category: str):
    def check(x: dict) -> bool:
        return x["category"] == category and x["confidence"] >= CONFIDENCE_THRESHOLD
    return check

production_router = RunnableBranch(
    (is_confident_category("billing"), billing_chain),
    (is_confident_category("technical"), technical_chain),
    fallback_chain,  # covers "general" AND low-confidence classifications
)

production_pipeline = RunnableLambda(classify_and_prepare) | production_router

response = production_pipeline.invoke(
    {"question": "I was double-billed on invoice #4471, can you check?"}
)
print(response)

This version quietly routes any low-confidence classification into the fallback path, even if the model technically picked "billing" — because a low-confidence "billing" guess is exactly the kind of case where you'd rather log it and give a careful general answer than confidently answer with the wrong specialist prompt.

Closing Thoughts

Router chains are a small idea with an outsized payoff: separate the decision of "who should answer this" from the actual answering, and every piece of your system gets easier to build, test, and improve independently. RunnableBranch gives you a clean, declarative way to express that decision when your categories are simple and fixed; a RunnableLambda-based dispatcher gives you more flexibility when the routing logic needs to do more than a chain of boolean checks. Structured output turns your classifier from a fragile string-matcher into something you can build real confidence thresholds and fallbacks around, and a properly designed default branch keeps the whole system honest about what it doesn't know.

If you want to go deeper into building these pipelines with real production patterns — structured output, agent handoffs, evaluation, and deployment — that's exactly what we cover hands-on in the LangChain Tutorial 2026 course on teachyou.ai, where we build routing systems like this one from scratch alongside full retrieval and agent pipelines.