LangChain Testing: Unit Testing Chains and Agents

Why LangChain Testing Feels Different From Normal Unit Testing

The first time most engineers try to write a test for a LangChain pipeline, they hit the same wall: the code calls a live model, the response is non-deterministic, the test takes eight seconds, and it costs money every time CI runs it. That's not a testing problem specific to you — it's the nature of building software on top of a probabilistic component. A RunnableSequence that pulls documents from a vector store, formats a prompt, and calls ChatOpenAI isn't a pure function. Feed it the same input twice and you can get two different outputs, two different token counts, and occasionally two different tool calls.

The temptation is to skip testing entirely and rely on "vibes" — running the chain manually in a notebook, eyeballing the output, and shipping it. That works until the chain has five steps, three conditional branches, and a tool-calling agent that occasionally decides to call the wrong tool. At that point, silent regressions are inevitable: someone changes a prompt template, an output parser starts throwing on a previously-valid format, or an agent's tool schema drifts out of sync with the actual function signature. Without tests, none of that surfaces until a user reports a broken answer in production.

LangChain testing isn't fundamentally different from testing any other system with an external dependency — you isolate the non-deterministic part, test your logic in isolation, and reserve a small number of expensive integration tests for the real model. The trick is knowing exactly where to draw that line, and LangChain actually gives you good primitives for it: fake chat models, runnable interfaces you can swap out, and callback hooks you can assert against. This article walks through unit testing chains, testing output parsers, testing agents and their tool-calling behavior, mocking retrievers, and building a pytest suite that runs in milliseconds instead of seconds.

The Three Layers Worth Testing Separately

Before writing a single test, it helps to mentally split a LangChain application into three layers, because each one needs a different testing strategy.

• Deterministic glue code: prompt templates, output parsers, retrievers that wrap a database query, custom Runnable classes, routing logic that picks between chains. This is regular Python — test it like regular Python, no LLM involved at all.
• The LLM call itself: the actual invoke() on a chat model. You almost never want to hit the real API in a unit test. Use a fake or mocked model.
• The orchestration behavior: does the chain call the retriever before the LLM? Does the agent stop after the tool returns a final answer? Does it retry on a parsing failure? This is where you assert on *sequence and structure*, not on exact wording.

Most of the value in a test suite comes from layers one and three. Layer two — "did GPT-4 give a good answer" — is an evaluation problem, not a unit test problem, and trying to assert exact LLM output text is how teams end up with brittle, constantly-breaking test suites. Keep that distinction in your head throughout: unit tests check that your code does what it's supposed to do with a given model response; evals check whether the model's response is actually good.

Setting Up a Fake Chat Model

LangChain ships FakeListChatModel and GenericFakeChatModel in langchain_core.language_models.fake_chat_models, which are built exactly for this. They implement the same Runnable interface as ChatOpenAI or ChatAnthropic, so any chain built with | composition works identically whether it's wired to the real model or the fake.

from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

def build_summary_chain(llm):
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Summarize the input in one sentence."),
        ("human", "{text}"),
    ])
    return prompt | llm | StrOutputParser()


def test_summary_chain_returns_parsed_string():
    fake_llm = FakeListChatModel(responses=["The article covers testing strategies for LangChain."])
    chain = build_summary_chain(fake_llm)

    result = chain.invoke({"text": "a long article about testing"})

    assert result == "The article covers testing strategies for LangChain."

Notice what this test actually verifies: that the prompt template renders without error, that the chain correctly pipes the LLM's AIMessage output into StrOutputParser, and that the final return type is a plain string. It says nothing about whether the summary is *good* — that's fine, that's not what this test is for.

FakeListChatModel cycles through a list of canned responses in order, which is useful when a test needs to simulate a multi-turn conversation or a chain that calls the LLM more than once. GenericFakeChatModel goes further and lets you pass a generator function, so you can make the fake respond differently based on the input it receives — handy for testing routing logic where the chain's next step depends on what the LLM said.

from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.messages import AIMessage

def fake_response_generator():
    yield AIMessage(content="ROUTE: billing")
    yield AIMessage(content="ROUTE: technical")

def test_router_chain_picks_billing_branch():
    fake_llm = GenericFakeChatModel(messages=fake_response_generator())
    router_chain = build_router_chain(fake_llm)

    result = router_chain.invoke({"query": "Why was I charged twice?"})

    assert result["route"] == "billing"

Testing Output Parsers in Isolation

Output parsers are some of the highest-value things to unit test because they're pure functions that fail in predictable, fixable ways — and because a parser exception is one of the most common causes of a production incident when the model's output format shifts even slightly.

If you're using PydanticOutputParser or a custom parser, test it directly against strings, without touching a model at all:

from pydantic import BaseModel
from langchain_core.output_parsers import PydanticOutputParser

class ExtractedInvoice(BaseModel):
    vendor: str
    amount: float
    due_date: str

parser = PydanticOutputParser(pydantic_object=ExtractedInvoice)

def test_parser_handles_well_formed_json():
    raw_output = '{"vendor": "Acme Corp", "amount": 129.99, "due_date": "2026-08-01"}'
    result = parser.parse(raw_output)

    assert result.vendor == "Acme Corp"
    assert result.amount == 129.99


def test_parser_raises_on_missing_field():
    raw_output = '{"vendor": "Acme Corp", "amount": 129.99}'

    with pytest.raises(Exception):
        parser.parse(raw_output)


def test_parser_handles_markdown_fenced_json():
    raw_output = '```json\n{"vendor": "Acme Corp", "amount": 129.99, "due_date": "2026-08-01"}\n```'
    result = parser.parse(raw_output)

    assert result.vendor == "Acme Corp"

That last test matters more than it looks — models frequently wrap JSON in markdown code fences even when told not to, and if your parser doesn't strip that, you'll get intermittent failures in production that are maddening to reproduce because they depend on model mood. Write the test now, before it happens to you at 2am.

If you've written a custom RunnableLambda as a parser — say, one that extracts a number from freeform text using regex — test the regex logic against a table of realistic and adversarial inputs:

import pytest

@pytest.mark.parametrize("raw_text,expected", [
    ("The answer is 42.", 42),
    ("I'd estimate around $3,200 total.", 3200),
    ("Confidence: 0.87", 0.87),
    ("No number here at all", None),
])
def test_extract_number_parser(raw_text, expected):
    assert extract_number(raw_text) == expected

Parametrized tests like this are where you should sink most of your effort, because they catch the edge cases that a single hand-written example never will — commas in numbers, currency symbols, decimals, and the "no match" case that your regex needs to fail gracefully on rather than throwing.

Unit Testing Multi-Step Chains With Runnable Composition

Chains built with LCEL (prompt | llm | parser | next_step) are just Runnable objects, and every Runnable supports invoke, batch, and stream. That means you can test a chain the same way regardless of how many steps it has, and you can substitute fakes at any point in the pipeline.

A retrieval-augmented chain is a good example, because it has both a retriever (deterministic, easy to fake) and an LLM call (non-deterministic, needs a fake model):

from langchain_core.runnables import RunnableLambda
from langchain_core.documents import Document

def build_rag_chain(retriever, llm):
    def format_docs(docs):
        return "\n\n".join(d.page_content for d in docs)

    prompt = ChatPromptTemplate.from_template(
        "Answer using only this context:\n{context}\n\nQuestion: {question}"
    )

    return (
        {
            "context": retriever | RunnableLambda(format_docs),
            "question": RunnableLambda(lambda x: x),
        }
        | prompt
        | llm
        | StrOutputParser()
    )


class FakeRetriever:
    def __init__(self, docs):
        self.docs = docs

    def invoke(self, query, config=None):
        return self.docs


def test_rag_chain_passes_retrieved_context_into_prompt():
    fake_docs = [Document(page_content="Refunds are processed within 5 business days.")]
    fake_retriever = FakeRetriever(fake_docs)
    fake_llm = FakeListChatModel(responses=["Refunds take 5 business days."])

    chain = build_rag_chain(fake_retriever, fake_llm)
    result = chain.invoke("How long do refunds take?")

    assert result == "Refunds take 5 business days."

This test wouldn't catch a bad answer from a real model, but it would catch a much more common and much more damaging bug class: a wiring mistake where the retriever's output never actually reaches the prompt, or the question key gets silently dropped because someone refactored the dict comprehension. Those are the bugs that unit tests are genuinely good at catching, and they happen far more often than people expect once a chain has more than three steps.

For chains with conditional branching — RunnableBranch or a custom router — test each branch independently by asserting the chain reaches the expected sub-chain given a particular fake response, exactly like the router example above. Don't try to write one giant test that covers every branch; write one small test per branch so failures point you directly at the broken path.

Testing Agents: Tool Calls Without Calling Real Tools

Agents add a layer of complexity because the model doesn't just produce text — it decides *which tool to call* and *with what arguments*, and your code has to execute that tool and feed the result back. The two things worth testing separately are: (1) does your agent correctly invoke the tool the model asked for, and (2) does your tool itself behave correctly given valid and invalid arguments.

Start with the tool in complete isolation — it's just a function, so test it like one:

from langchain_core.tools import tool

@tool
def get_order_status(order_id: str) -> str:
    """Look up the shipping status of an order by its ID."""
    order = database.find_order(order_id)
    if order is None:
        return f"No order found with ID {order_id}"
    return f"Order {order_id} is {order.status}"


def test_get_order_status_returns_status_for_known_order(mock_database):
    mock_database.find_order.return_value = FakeOrder(status="shipped")

    result = get_order_status.invoke({"order_id": "ORD-1001"})

    assert "shipped" in result


def test_get_order_status_handles_unknown_order(mock_database):
    mock_database.find_order.return_value = None

    result = get_order_status.invoke({"order_id": "ORD-9999"})

    assert "No order found" in result

Now test the agent's decision loop using FakeListChatModel seeded with a tool-call response. LangChain models represent tool calls as structured AIMessage.tool_calls, so the fake model needs to emit that shape:

from langchain_core.messages import AIMessage

def test_agent_calls_get_order_status_tool(monkeypatch):
    tool_call_message = AIMessage(
        content="",
        tool_calls=[{
            "name": "get_order_status",
            "args": {"order_id": "ORD-1001"},
            "id": "call_1",
        }],
    )
    final_message = AIMessage(content="Your order ORD-1001 has shipped.")

    fake_llm = FakeListChatModel(responses=[tool_call_message, final_message])
    agent_executor = build_agent(fake_llm, tools=[get_order_status])

    result = agent_executor.invoke({"input": "Where is my order ORD-1001?"})

    assert "shipped" in result["output"]

If your agent runner exposes callbacks or intermediate steps (most AgentExecutor configurations do via return_intermediate_steps=True), assert on those directly rather than only the final string — it's a much stronger test:

def test_agent_invokes_correct_tool_with_correct_args():
    fake_llm = FakeListChatModel(responses=[tool_call_message, final_message])
    agent_executor = build_agent(fake_llm, tools=[get_order_status], return_intermediate_steps=True)

    result = agent_executor.invoke({"input": "Where is my order ORD-1001?"})

    action, observation = result["intermediate_steps"][0]
    assert action.tool == "get_order_status"
    assert action.tool_input == {"order_id": "ORD-1001"}

This catches a specific and common bug: an agent that calls the *right* tool but with the *wrong* arguments — say, passing a customer name where an order ID was expected because the tool's docstring was ambiguous. Testing only the final output would let that bug through if the LLM happens to recover gracefully; testing intermediate steps catches it every time.

Mocking Retrievers and Vector Stores

Real vector stores are slow to spin up in a test suite and introduce nondeterminism from embedding similarity scoring. For unit tests, skip the actual embedding model and vector database entirely and use an in-memory stand-in or a simple mock.

from unittest.mock import MagicMock
from langchain_core.documents import Document

def test_retriever_is_called_with_the_raw_query(mocker):
    mock_vectorstore = MagicMock()
    mock_vectorstore.similarity_search.return_value = [
        Document(page_content="Password resets are self-service via the settings page."),
    ]

    retriever = mock_vectorstore.as_retriever()
    docs = retriever.invoke("How do I reset my password?")

    mock_vectorstore.similarity_search.assert_called()
    assert len(docs) == 1
    assert "self-service" in docs[0].page_content

For a slightly higher-fidelity test that still avoids network calls or a real database, LangChain's in-memory vector store (InMemoryVectorStore) combined with a deterministic fake embeddings class lets you test actual similarity search logic without any external dependency:

from langchain_core.vectorstores import InMemoryVectorStore
from langchain_core.embeddings import DeterministicFakeEmbedding

def test_similarity_search_returns_relevant_document():
    embeddings = DeterministicFakeEmbedding(size=64)
    store = InMemoryVectorStore(embeddings)
    store.add_documents([
        Document(page_content="Refund policy: 30 days, no questions asked."),
        Document(page_content="Our office is open 9 to 5 on weekdays."),
    ])

    results = store.similarity_search("refund", k=1)

    assert "Refund policy" in results[0].page_content

This is a good middle ground: it exercises real Runnable interfaces and real similarity search math without touching an external API, network, or paid embedding call, which means it runs in milliseconds and can execute thousands of times in CI without cost.

Structuring the Test Suite With pytest Fixtures

Once you have more than a handful of chains and tools, put shared fakes into conftest.py so every test file can reuse them instead of re-declaring boilerplate:

# conftest.py
import pytest
from langchain_core.language_models.fake_chat_models import FakeListChatModel

@pytest.fixture
def fake_llm_factory():
    def _make(responses):
        return FakeListChatModel(responses=responses)
    return _make


@pytest.fixture
def sample_documents():
    from langchain_core.documents import Document
    return [
        Document(page_content="Shipping takes 3-5 business days.", metadata={"source": "faq"}),
        Document(page_content="Returns are accepted within 30 days.", metadata={"source": "faq"}),
    ]

# test_chains.py
def test_shipping_question_uses_correct_document(fake_llm_factory, sample_documents):
    fake_llm = fake_llm_factory(["Shipping takes 3-5 business days."])
    retriever = FakeRetriever(sample_documents)
    chain = build_rag_chain(retriever, fake_llm)

    result = chain.invoke("How long does shipping take?")

    assert "3-5 business days" in result

Organize the suite into three directories that mirror the three layers discussed earlier: tests/unit/parsers, tests/unit/chains, tests/unit/agents for the fast, fake-model tests that run on every commit, and a separate tests/integration directory, marked with a pytest marker like @pytest.mark.integration, for the small number of tests that hit a real model to catch prompt-format drift. Run the unit suite on every push and gate the integration suite to nightly runs or pre-release checks, since those cost real API tokens and take real wall-clock time.

# pytest.ini or pyproject.toml
[tool.pytest.ini_options]
markers = [
    "integration: tests that call a real LLM API (slow, costs money)",
]

# fast feedback loop, every commit
pytest tests/unit -x -q

# slower, gated to CI nightly job or before a release
pytest tests/integration -m integration

Common Pitfalls to Avoid

• Asserting exact LLM wording in unit tests. Even with a fake model this can happen if you copy real model output into the fake's canned response and then assert against it verbatim elsewhere — it's fragile the moment anyone touches the prompt. Assert on structure, keys, and substrings, not full sentences, unless the fake response itself is the thing under test.
• Skipping tool argument validation. A tool decorated with @tool should validate its own inputs — test that it rejects malformed arguments rather than trusting the model to always produce well-formed calls, because it won't always.
• Testing only the happy path for output parsers. Malformed JSON, truncated responses, and markdown fences around structured output are not edge cases in production — they're routine. Write the failure-mode tests before you ship, not after the first incident.
• Running integration tests on every CI push. This burns API budget and slows the feedback loop for a wrong reason — most bugs are in your glue code, not in the model's ability to respond, and your fast unit suite already covers that.
• Forgetting to test retries and fallbacks. If your chain uses .with_retry() or .with_fallbacks(), write a test where the fake model raises an exception on the first call and succeeds on the second, and assert the retry actually happened rather than assuming the wrapper works as documented.

Wrapping Up

Testing LangChain code well comes down to a single habit: keep the non-deterministic model call as small and isolated as possible, and put your testing effort into everything around it — prompt construction, output parsing, tool argument handling, retrieval wiring, and agent decision loops. FakeListChatModel and GenericFakeChatModel let you write fast, deterministic, free unit tests that run on every commit, while a much smaller set of real-model integration tests catches genuine prompt or model-behavior drift on a slower cadence. Do this consistently and a five-step RAG chain or a multi-tool agent stops being a black box you're afraid to touch — it becomes ordinary software with an ordinary test suite, refactorable with confidence instead of dread.

If you want to go deeper — building full test suites for production agent systems, wiring CI pipelines for LLM applications, and structuring evaluation frameworks alongside unit tests — that's exactly what we cover hands-on in the LangChain Tutorial 2026 course on teachyou.ai, with real chains, real agents, and real test suites built from scratch.