LangChain

LangChain Chat Models vs LLMs: Understanding the Interface Difference

Pramod Dutta · Jul 2, 2026 · 14 min read

Why This Confuses Almost Everyone Who Starts With LangChain

If you have spent any time inside the LangChain codebase, you have probably hit a moment of quiet confusion. You import something called OpenAI from langchain_openai, and then you also see ChatOpenAI sitting right next to it. Both talk to OpenAI. Both return text. Both feel like they should be interchangeable. They are not, and the difference is not cosmetic.

This is one of those places where LangChain's abstraction layer is doing real work under the hood, and if you do not understand what that work is, you will eventually write code that either breaks in production or quietly produces worse results than it should. The distinction between Chat Models and LLMs in LangChain reflects a deeper split in how modern language model providers expose their APIs, and once you understand that split, half of the "why does this not work the way I expect" confusion disappears.

In this article we are going to open up both interfaces, look at the actual message formats they expect, trace through the code paths, and build a mental model that will save you hours of debugging later. This matters more than it sounds, because almost every LangChain chain, agent, and RAG pipeline you will build in a real project routes through one of these two interfaces, and picking the wrong one — or misunderstanding how the right one behaves — is a common source of subtle bugs.

The Historical Reason This Split Exists

To understand why LangChain has two separate abstractions, it helps to remember how the underlying model APIs evolved.

Early large language model APIs, like the original GPT-3 completion endpoint, worked on a very simple contract: you send a single string of text, and the model returns a continuation of that string. There was no concept of "roles," no system prompt, no distinction between what the user said and what the assistant said. It was pure text-in, text-out. This is the completion paradigm, and it is what LangChain's LLM base class was originally built to wrap.

Then came instruction-tuned and chat-tuned models. OpenAI's gpt-3.5-turbo and everything since has used a fundamentally different API shape: instead of a single string, you send a list of messages, each tagged with a role — system, user, assistant, and later tool. The model was fine-tuned specifically to understand conversational turn-taking, and the API reflects that. This is the chat paradigm, and it is what LangChain's ChatModel base class wraps.

So the split in LangChain is not an arbitrary design decision by the maintainers — it is a direct mirror of a split that already existed in the underlying provider APIs. LangChain just gives each paradigm its own class hierarchy so that the input and output types match what the API actually expects.

The LLM Interface: Plain Text In, Plain Text Out

Let's start with the older, simpler interface. In LangChain, classes that inherit from BaseLLM (commonly just called "LLMs") accept a plain string as input and return a plain string as output.

from langchain_openai import OpenAI

llm = OpenAI(model="gpt-3.5-turbo-instruct", temperature=0.7)

response = llm.invoke("Write a one-line tagline for a coffee shop.")
print(response)
print(type(response))  # <class 'str'>

Notice what is happening here. There is no role structure. There is no system prompt as a distinct concept — if you want the model to behave a certain way, you have to bake instructions directly into the string you send. The invoke method takes a string and returns a string. That is the entire contract.

Under the hood, BaseLLM implementations call completion-style endpoints. Fewer and fewer providers still expose these endpoints for their frontier models, because completion-style models are largely legacy at this point. Most providers have moved their newest, most capable models exclusively to chat-style endpoints. This is important context: the LLM interface in LangChain still exists and still works, but it is increasingly wrapping older or more niche models rather than the flagship ones.

Here is what the internal call structure roughly looks like when you trace through BaseLLM:

class BaseLLM(BaseLanguageModel):
    def invoke(self, input: str, config=None, **kwargs) -> str:
        # internally calls self._generate or self._call
        # sends a raw prompt string to the provider
        # returns raw completion text
        result = self._call(prompt=input, **kwargs)
        return result

The key thing to internalize: the LLM interface has no native concept of conversation. If you want multi-turn behavior with an LLM, you have to manually concatenate the conversation history into a single string yourself, using whatever prompt format the underlying model was trained on. LangChain will not do this for you, because the interface simply was not designed with turn-based roles in mind.

The Chat Model Interface: Structured Messages In, a Message Out

Now compare that to BaseChatModel, which is the parent class for things like ChatOpenAI, ChatAnthropic, and ChatGoogleGenerativeAI. Instead of a string, the input is a list of message objects, each carrying an explicit role.

from langchain_anthropic import ChatAnthropic
from langchain_core.messages import SystemMessage, HumanMessage

chat_model = ChatAnthropic(model="claude-opus-4-6", temperature=0.7)

messages = [
    SystemMessage(content="You are a concise marketing copywriter."),
    HumanMessage(content="Write a one-line tagline for a coffee shop."),
]

response = chat_model.invoke(messages)
print(response.content)
print(type(response))  # <class 'langchain_core.messages.ai.AIMessage'>

Two things changed compared to the LLM example. First, the input is a structured list of typed message objects, not a raw string. Second, the output is not a plain string — it is an AIMessage object, which carries .content but can also carry .tool_calls, .response_metadata, .usage_metadata, and other structured fields depending on what the model returned.

This is not a minor packaging difference. It reflects the fact that chat models were trained on structured conversational data, and the API contract preserves that structure end-to-end. When you send a SystemMessage, it maps directly onto the "system" role the model was fine-tuned to recognize. When the model wants to call a tool, that intent comes back as a tool_calls field on the AIMessage, not as text you have to parse out of a string.

Message Types You Will Actually Use

LangChain defines a small family of message classes in langchain_core.messages, and knowing them cold will make chat model code much easier to read and write.

SystemMessage — sets behavior, tone, or constraints for the whole conversation. Sent once, usually first.
HumanMessage — represents user input. Can be plain text or, for multimodal models, a list of content blocks (text plus images).
AIMessage — represents what the model said. This is also the type returned by invoke().
ToolMessage — represents the result of a tool call, tied back to the model's request via a tool_call_id.
FunctionMessage — an older, mostly deprecated precursor to ToolMessage, kept around for backward compatibility.

Here is a fuller multi-turn example that shows how a real conversation, including a tool call round-trip, gets represented:

from langchain_core.messages import (
    SystemMessage, HumanMessage, AIMessage, ToolMessage
)

conversation = [
    SystemMessage(content="You are a helpful assistant with access to a calculator tool."),
    HumanMessage(content="What is 482 * 17?"),
    AIMessage(
        content="",
        tool_calls=[{
            "name": "calculator",
            "args": {"expression": "482 * 17"},
            "id": "call_001",
        }],
    ),
    ToolMessage(content="8194", tool_call_id="call_001"),
]

response = chat_model.invoke(conversation)
print(response.content)  # e.g. "482 * 17 is 8194."

Try building that same round-trip with a plain LLM interface and you will feel the pain immediately — you would need to hand-roll a prompt template that encodes roles as text markers, hope the underlying model was trained to recognize your chosen format, and manually parse any "tool call" out of free-form text using regex or a fragile JSON-in-string convention. The chat interface makes this a first-class, structured operation instead of a string-parsing exercise.

Why Tool Calling Basically Requires Chat Models

This point deserves its own section because it trips up a lot of people building agents. LangChain's tool-calling and agent frameworks — bind_tools, create_tool_calling_agent, most of LangGraph's prebuilt agent nodes — are built almost entirely on top of BaseChatModel, not BaseLLM.

from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"It is sunny in {city}."

model = ChatOpenAI(model="gpt-4o")
model_with_tools = model.bind_tools([get_weather])

response = model_with_tools.invoke("What's the weather in Lisbon?")
print(response.tool_calls)
# [{'name': 'get_weather', 'args': {'city': 'Lisbon'}, 'id': '...', 'type': 'tool_call'}]

bind_tools works because chat model providers expose a dedicated parameter in their API for passing tool/function schemas, and the model returns structured tool-call data in the response rather than text. BaseLLM implementations, wrapping older completion endpoints, generally have no equivalent mechanism — there is no "tools" parameter to bind to on a raw completion call. Some legacy setups fake this with prompt-based "ReAct"-style parsing, where the model is instructed to output text like Action: search\nAction Input: ... and LangChain regex-parses it back out. It works, but it is noticeably more brittle than native tool calling, and it is largely a legacy pattern at this point.

If you are building anything agentic in 2026 — and most serious LangChain projects at this point are — you almost certainly want ChatModel, not LLM.

Streaming, Batching, and Async: Both Support Them, Differently Shaped

Both interfaces implement LangChain's Runnable protocol, so both support .invoke(), .stream(), .batch(), .ainvoke(), .astream(), and .abatch(). The difference is in what gets streamed.

# Chat model streaming yields AIMessageChunk objects
for chunk in chat_model.stream(messages):
    print(chunk.content, end="", flush=True)

# LLM streaming yields raw string chunks
for chunk in llm.stream("Tell me a short story about a lighthouse."):
    print(chunk, end="", flush=True)

With a chat model, each streamed chunk is an AIMessageChunk, which supports being added together (chunk1 + chunk2) to progressively reconstruct the full message, including partial tool-call arguments as they stream in. With an LLM, each chunk is just a string fragment. This matters if you are building a UI that needs to show a tool call being "typed out" progressively, or if you need structured metadata (like token usage) attached to the stream itself — chat models expose that naturally through the message chunk objects, while LLMs generally do not.

A Common Trap: Assuming You Can Swap One For the Other

A mistake I see often in code review: someone writes a chain against ChatOpenAI, then later swaps in a different model class expecting it to behave identically, without checking whether that class extends BaseChatModel or BaseLLM. LangChain's naming conventions do not always make this obvious at a glance in third-party integration packages, so it is worth actually checking.

from langchain_core.language_models import BaseChatModel, BaseLLM

def describe_interface(model) -> str:
    if isinstance(model, BaseChatModel):
        return "chat model: expects list[BaseMessage], returns AIMessage"
    elif isinstance(model, BaseLLM):
        return "llm: expects str, returns str"
    return "unknown interface"

print(describe_interface(chat_model))  # chat model: ...
print(describe_interface(llm))         # llm: ...

This kind of check is cheap insurance. If you are building a wrapper function, a factory, or a config-driven model loader that might return either type depending on a settings file, guard against the mismatch early rather than discovering it three layers deep in a stack trace when .invoke() receives a string it was not expecting.

Another subtlety: ChatModel.invoke() is actually somewhat forgiving about input — you can often pass it a plain string and LangChain will silently wrap it into a HumanMessage for convenience.

# This works — LangChain coerces the string into a HumanMessage internally
response = chat_model.invoke("What's the capital of Portugal?")
print(response.content)

This convenience can mask the underlying difference and lull people into thinking the two interfaces are more similar than they are. The coercion only goes one way, though — you cannot pass a list of BaseMessage objects into an LLM's invoke() and expect it to do anything sensible with the role structure, because the completion endpoint underneath has no concept of roles to map them onto.

Prompt Templates Also Branch on This Distinction

LangChain's prompt template classes mirror this same split. PromptTemplate produces plain strings meant for LLM classes. ChatPromptTemplate produces a list of messages meant for ChatModel classes.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a terse technical writer."),
    ("human", "Summarize this in one sentence: {text}"),
])

chain = prompt | chat_model
result = chain.invoke({"text": "LangChain provides two model interfaces: LLM and ChatModel."})
print(result.content)

ChatPromptTemplate.from_messages accepts tuples of (role, template_string), and when the chain runs, it formats those into the corresponding message objects and hands them to the chat model. This is the idiomatic modern pattern in LangChain — piping a ChatPromptTemplate directly into a ChatModel using the | operator, letting the Runnable interface handle the plumbing. If you find yourself instead building a single giant f-string with manual role markers baked in and feeding it to a plain LLM, that is usually a sign you are fighting the framework rather than working with it.

Which One Should You Actually Use in 2026

For nearly all new projects, the answer is: use ChatModel. Here is the reasoning laid out plainly:

Almost every frontier model provider — Anthropic, OpenAI, Google — exposes their best models exclusively through chat-style APIs now. There often isn't a completion endpoint to wrap even if you wanted one.
Native tool calling, structured output parsing, and multimodal input (images, documents) are built around the message-based interface, not the string-based one.
LangGraph, which has become the standard way to build stateful agents in the LangChain ecosystem, expects chat models as its default building block.
The message-object output (AIMessage) gives you structured access to metadata — token usage, stop reason, tool calls — that a plain string from an LLM simply cannot carry.

The LLM interface is not dead, but its realistic use cases have narrowed. You will still see it for legacy completion-style models, some fine-tuned open-source models served through simple text-completion wrappers, or narrow internal tools where you genuinely just want raw text continuation with no conversational structure at all. If you are starting a new project today, defaulting to ChatModel and only reaching for LLM when you have a specific reason is the right instinct.

Debugging Checklist When Things Feel "Off"

When a LangChain chain misbehaves in a way that smells like an interface mismatch, run through this quickly:

Print type(model) and check its MRO for BaseChatModel versus BaseLLM.
Print type(response) after invoke() — a bare str means you are on the LLM path, an AIMessage means you are on the chat path.
If tool calls are not showing up, confirm the model is a ChatModel and that you actually called .bind_tools() before invoking.
If you're piping a ChatPromptTemplate into a model, confirm the model isn't a plain LLM expecting a string — the pipe will still technically run, but role information can get flattened or dropped in unexpected ways.
Check whether the provider integration package you're using has separate classes (OpenAI versus ChatOpenAI) and make sure you imported the one you meant to.

Most "why isn't my agent calling tools" or "why does my system prompt get ignored" questions trace back to one of these five checks.

Wrapping Up

The split between ChatModel and LLM in LangChain is not framework bloat — it is an honest reflection of two genuinely different API contracts that model providers expose. LLM wraps the older completion paradigm: string in, string out, no role structure, no native tool calling. ChatModel wraps the modern chat paradigm: a list of typed, role-tagged messages in, a structured AIMessage out, with native support for system prompts, tool calls, and streaming metadata.

Once this distinction clicks, a lot of LangChain's API surface stops feeling arbitrary. You start understanding why bind_tools lives where it does, why ChatPromptTemplate and PromptTemplate are separate classes, and why nearly every modern tutorial you find reaches for ChatOpenAI or ChatAnthropic rather than their non-chat counterparts. Get comfortable with BaseChatModel, its message types, and how it interacts with tool binding and streaming, and you will have covered the interface that underpins almost every serious LangChain and LangGraph application being built right now.

If you want to go deeper into this — building real agents, wiring up tool calling end to end, working through RAG pipelines and multi-step LangGraph workflows with proper debugging habits — that is exactly what we cover hands-on in the LangChain Tutorial 2026 course on teachyou.ai, starting from these core interface fundamentals and building up to production-grade agent systems.