LangGraph Deployment: Serving a Graph as a Production API
From Notebook to Endpoint: Why This Transition Breaks Most Teams
You built a graph. It runs beautifully in a Jupyter cell — nodes fire in order, tool calls resolve, the final state prints exactly what you expected. Then someone asks the question that ends the honeymoon: "Can we hit this from the mobile app?"
Suddenly your elegant graph.invoke() call needs to survive concurrent requests, resume after a server restart, stream tokens to a UI, remember conversation history across sessions, and fail gracefully when a downstream API times out. None of that is exposed by .invoke(). It's infrastructure, and infrastructure is where most agent projects quietly stall.
This is the deployment gap: the distance between "the graph runs" and "the graph is a service." LangGraph actually gives you two credible paths across that gap. One is LangGraph Platform, a deployment target purpose-built for graphs, with baked-in persistence, streaming, and horizontal scaling. The other is rolling your own FastAPI service around a checkpointer, which costs more engineering time but keeps you in full control of infrastructure, auth, and hosting.
This article walks through both, with working code, so you can pick the one that matches your team's constraints instead of guessing.
What "Production" Actually Requires From a Graph Server
Before comparing options, be precise about what changes when a graph goes from notebook to API. Four requirements show up in almost every production deployment:
- Persistence across requests. A user sends message one, then message three minutes later. The graph needs to remember the state from message one without you manually re-serializing it into the request body.
- Concurrency safety. Ten users invoke the graph at the same time. Each needs an isolated thread of execution and state, not a shared global that gets clobbered.
- Streaming output. Waiting 8 seconds for a full response feels broken to end users. You need token-level or step-level streaming over HTTP.
- Recoverability. A node crashes, a pod restarts, a rate limit trips mid-run. The graph should be able to resume from its last checkpoint instead of forcing the user to start over.
graph.invoke(input) in a script solves none of these by default. Everything below is about closing that gap deliberately, with a checkpointer doing the heavy lifting for persistence and recovery, and either LangGraph Platform or an ASGI server doing the heavy lifting for concurrency and streaming.
Path One: LangGraph Platform (the Managed Route)
LangGraph Platform is LangGraph's own deployment product: you point it at a graph definition, and it gives you a versioned, authenticated REST API with built-in threads, streaming, and horizontal scaling, without you writing a web server.
The unit of deployment is a langgraph.json config file plus your graph module. Here's a minimal one:
{
"dependencies": ["."],
"graphs": {
"support_agent": "./src/agent_graph.py:graph"
},
"env": ".env"
}That single line tells the platform where to find a compiled graph object named graph inside agent_graph.py. The graph itself looks like any LangGraph graph you'd build locally:
# src/agent_graph.py
from typing import TypedDict, Annotated
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph.checkpoint.memory import MemorySaver
from langchain_anthropic import ChatAnthropic
class AgentState(TypedDict):
messages: Annotated[list, add_messages]
llm = ChatAnthropic(model="claude-sonnet-4-5")
def call_model(state: AgentState):
response = llm.invoke(state["messages"])
return {"messages": [response]}
builder = StateGraph(AgentState)
builder.add_node("agent", call_model)
builder.add_edge(START, "agent")
builder.add_edge("agent", END)
# LangGraph Platform manages its own checkpointer in production,
# but compiling with one keeps local `langgraph dev` runs stateful too.
graph = builder.compile(checkpointer=MemorySaver())Locally, langgraph dev spins up this graph behind a dev server with hot reload, so you can hit it with curl before touching real infrastructure:
langgraph dev
# serves on http://localhost:2024 by default
curl -s http://localhost:2024/threads -X POST \
-H "Content-Type: application/json" -d '{}'That call creates a thread — Platform's abstraction for a persistent conversation. Every subsequent run against that thread ID replays from the last checkpoint automatically; you never manually load or save state.
curl -s http://localhost:2024/threads/<thread_id>/runs/stream \
-H "Content-Type: application/json" \
-d '{
"assistant_id": "support_agent",
"input": {"messages": [{"role": "user", "content": "Refund status for order 4471?"}]}
}'That endpoint streams Server-Sent Events back immediately: node-start events, token deltas from the model, and a final values event with the complete state. Your frontend subscribes once and renders incrementally instead of polling.
When you're ready to ship, langgraph deploy (or pushing to a connected repo, depending on your Platform plan) builds a container from your langgraph.json and dependency file, and gives you a hosted URL with API-key auth, autoscaling, and a built-in trace view for every run. You don't provision servers, write Dockerfiles, or manage a Postgres instance for checkpoints — Platform's managed persistence layer handles that.
The trade-off is control. You're deploying into LangGraph's infrastructure model, so custom auth flows, non-standard networking, or exotic scaling rules need to fit inside what Platform exposes. For most product teams shipping a chat-style or agentic feature, that's a fair trade for not building a state management layer from scratch.
Path Two: Self-Hosting With FastAPI and a Checkpointer
If you need to embed the graph inside an existing service, control the exact hosting environment, or avoid a managed dependency, you can serve LangGraph yourself. The graph code doesn't change — only the layer around it does.
Start with a durable checkpointer instead of the in-memory one. MemorySaver loses all state on restart, which is fine for a demo and a liability in production. Swap it for the Postgres checkpointer:
# src/agent_graph.py
from langgraph.checkpoint.postgres import PostgresSaver
from psycopg_pool import ConnectionPool
DB_URI = "postgresql://user:pass@localhost:5432/langgraph_state"
pool = ConnectionPool(conninfo=DB_URI, max_size=20, kwargs={"autocommit": True})
checkpointer = PostgresSaver(pool)
checkpointer.setup() # creates checkpoint tables on first run
graph = builder.compile(checkpointer=checkpointer)checkpointer.setup() is idempotent and creates the schema LangGraph needs to store state snapshots per thread. Run it once at startup; after that, every graph.invoke() or graph.stream() call that includes a thread_id in its config automatically reads and writes checkpoints from Postgres.
Now wrap the graph in a FastAPI app. The key design decision here is exposing a streaming endpoint, since that's what most production UIs actually need:
# src/main.py
import json
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from src.agent_graph import graph
app = FastAPI(title="Support Agent API")
class ChatRequest(BaseModel):
thread_id: str
message: str
@app.post("/chat/stream")
async def chat_stream(req: ChatRequest):
config = {"configurable": {"thread_id": req.thread_id}}
inputs = {"messages": [{"role": "user", "content": req.message}]}
async def event_generator():
async for event in graph.astream_events(inputs, config, version="v2"):
if event["event"] == "on_chat_model_stream":
chunk = event["data"]["chunk"].content
if chunk:
yield f"data: {json.dumps({'token': chunk})}\n\n"
elif event["event"] == "on_chain_end" and event["name"] == "LangGraph":
yield f"data: {json.dumps({'done': True})}\n\n"
return StreamingResponse(event_generator(), media_type="text/event-stream")
@app.get("/health")
async def health():
return {"status": "ok"}astream_events is doing the real work here: it emits a fine-grained event stream for every node execution and every token the model produces, tagged by event type. Filtering for on_chat_model_stream gives you token-level streaming without writing your own buffering logic; filtering for the graph's on_chain_end tells you the whole run has finished, which is your signal to close the stream.
Because thread_id is threaded through config on every call, concurrent requests with different thread IDs are automatically isolated — the checkpointer keys state by thread, so ten simultaneous users get ten independent conversations without any manual locking in your route handler.
Run it with an ASGI server that supports multiple workers:
uvicorn src.main:app --host 0.0.0.0 --port 8000 --workers 4Because state lives in Postgres rather than process memory, those four workers can share load without stepping on each other's conversations, and a worker restart doesn't lose any user's history.
Handling Long-Running and Human-in-the-Loop Graphs
Some graphs don't finish in one request-response cycle. A node might pause for human approval, wait on an external webhook, or run a task that takes minutes. LangGraph's answer to this is interrupts combined with checkpointing — the graph literally stops mid-execution and waits, and your API layer just needs to expose a way to resume it.
from langgraph.types import interrupt, Command
def request_approval(state: AgentState):
decision = interrupt({"question": "Approve refund of $84.00?"})
if decision == "approved":
return {"messages": [{"role": "system", "content": "Refund processed."}]}
return {"messages": [{"role": "system", "content": "Refund denied."}]}When the graph hits interrupt(), execution halts and the current state is checkpointed exactly where it stopped. Your API returns that pending state to the client instead of a final answer:
@app.post("/chat/resume")
async def resume(req: ChatRequest):
config = {"configurable": {"thread_id": req.thread_id}}
result = await graph.ainvoke(Command(resume=req.message), config)
return {"state": result}The client calls /chat initially, gets back a pending-approval state, shows a UI prompt, and then calls /chat/resume with the human's decision once it's available — potentially hours later, on a different server instance, because the checkpoint in Postgres (or Platform's managed store) is what carries the state forward, not anything held in process memory.
This pattern is the difference between a graph that merely runs and one that can model real workflows: approvals, escalations, waiting on a third-party callback. It's also where self-hosting with a durable checkpointer earns its keep — without persisted state, "wait for a human, possibly for hours" simply isn't representable.
Observability: Tracing Every Node in Production
Once a graph is live, "it worked in my test" stops being useful information. You need visibility into which node ran, what it received, what it returned, and how long it took, for every request, not just the ones you manually inspect.
LangSmith integrates directly with LangGraph and needs almost no code to switch on:
import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_PROJECT"] = "support-agent-prod"
os.environ["LANGCHAIN_API_KEY"] = "ls__..."With those three environment variables set, every graph.invoke(), graph.stream(), or astream_events() call automatically emits a full trace: each node's input and output state, every LLM call with token counts and latency, and any tool invocations nested inside a node. If a customer reports a wrong answer, you pull up the exact run by thread ID and see precisely which node produced the bad output, rather than guessing from logs.
For teams that can't send traces to an external service, the same event stream that powers /chat/stream above can be piped into your existing logging or metrics stack — astream_events gives you a event-type field you can switch on to increment counters (nodes_executed, llm_calls, tool_errors) per request, which is often enough to catch regressions before a customer does.
Deployment Checklist: What to Verify Before Going Live
Regardless of which path you take, a handful of checks separate a demo from something you'd trust with real traffic:
- Checkpointer is durable, not in-memory.
MemorySavershould never reach a production deployment — confirm you're on Postgres, SQLite-with-persistent-disk, or Platform's managed store. - Every request carries a `thread_id`. Forgetting this silently creates a new, stateless conversation on every call — a common bug that looks like "the agent has no memory."
- Streaming is wired end-to-end. Test with
curl -Nor a real browserEventSource, not just a script that buffers the whole response — buffering can mask a broken stream. - Timeouts and retries are set on external calls. A node calling a third-party API without a timeout can hang a worker indefinitely; wrap those calls explicitly.
- Interrupts have a resume path tested end-to-end. If any node uses
interrupt(), verify the resume endpoint works after a real process restart, not just within the same run. - Tracing is on before, not after, the first incident. Turning on LangSmith or event logging after something breaks means you have no trace of the thing that broke.
Choosing Between Platform and Self-Hosted FastAPI
There's no universally correct answer here, but the decision usually comes down to three questions.
- Do you need this live fast, with minimal ops burden? LangGraph Platform gets you a streaming, persistent, authenticated API from a
langgraph.jsonfile and adeploycommand. That's the right call for most teams shipping a single agentic feature. - Does the graph need to live inside an existing service, VPC, or auth system you don't control? Self-hosting with FastAPI and a Postgres checkpointer gives you that control, at the cost of owning the deployment, scaling, and monitoring yourself.
- Are you still iterating on graph structure daily?
langgraph dev's hot reload is a genuinely faster local loop than restarting a FastAPI process on every graph change, even if you plan to self-host the final version.
Many teams actually land on a hybrid: prototype and iterate using langgraph dev, then either deploy to Platform directly, or take the same graph module and mount it inside a FastAPI app they already run, once the interface has stabilized. Because the graph object itself doesn't change between these paths — only the layer wrapping it does — that migration is cheap. Nothing above locks you into a decision you made in week one.
Closing: The Graph Was Never the Hard Part
Building the graph — nodes, edges, conditional routing, tool calls — is the part every tutorial covers, and it's genuinely the smaller half of the work. Making that graph survive concurrent users, server restarts, long waits for human approval, and the eventual "why did it say that" support ticket is where production engineering actually happens. Checkpointers, thread IDs, streaming events, and interrupts are the primitives LangGraph gives you to do that without inventing your own state machine on top of a state machine.
If you want to go through this end-to-end — building a graph, adding a Postgres checkpointer, deploying it both to LangGraph Platform and as a self-hosted FastAPI service, and wiring up LangSmith tracing on a real project — that full walkthrough is exactly what we cover in the LangGraph Tutorial course on teachyou.ai, with working code you can adapt directly into your own agent's deployment.
BootcampA 30-day guided bootcamp: build, harden and ship a production autonomous agent from scratch.
AI AgentsUnderstand how AI agents really work: the loop, the tools, the memory, and why most agent projects fail.