LangSmith vs Langfuse vs Helicone: LLM Observability Compared

Your agent worked in the demo. Then it shipped, a user typed something you never tested, three tool calls happened in a sequence you didn't expect, and the final answer was subtly wrong. Now you're staring at a log line with a 4,000-token prompt and no idea which of the six intermediate steps introduced the problem. This is the moment every team discovers that "just add some print statements" was never a monitoring strategy — it was a placeholder for one. LLM observability tools exist specifically for this moment, and the three names that come up most are LangSmith, Langfuse, and Helicone. They solve overlapping problems but they are not interchangeable, and picking the wrong one means either paying for depth you don't need or hitting a wall right when your app gets complex enough to need it.

This comparison is written from the perspective of someone who has wired all three into real projects, not from a feature-matrix scrape. We'll go tool by tool, then compare them directly on the axes that actually decide this for a team: framework lock-in, self-hosting versus managed, cost-tracking granularity, and whether you need deep trace trees or just a number on a dashboard.

The core problem observability solves

Before comparing tools, it's worth being precise about what "LLM observability" actually means, because it's different from traditional APM. In a normal web app, a request comes in, hits a few services, and returns — the trace is largely mechanical. In an LLM application, a single user request can fan out into a prompt chain: a retrieval step, a re-ranking step, a first LLM call that decides which tool to invoke, the tool call itself, a second LLM call that synthesizes the result, and maybe a third call that grades its own output. Each of those steps has its own latency, its own token cost, and its own failure mode. A generic error tracker sees one HTTP request. An LLM observability tool needs to see the whole tree.

On top of that, LLM apps fail silently. The code doesn't throw an exception when the model hallucinates a citation or ignores half the system prompt — it just returns a plausible-looking string. So observability here isn't only about latency and uptime, it's about capturing enough of the input/output pairs, intermediate reasoning, and metadata that you can go back later and evaluate quality, not just health. That's the job all three tools are trying to do, with different philosophies about how much structure to impose and how much of the stack you have to own.

LangSmith

LangSmith is built by the LangChain team, and it shows. If your application is built with LangChain or LangGraph, LangSmith traces show up with almost zero configuration — chains, agents, tool calls, and retries are automatically decomposed into a trace tree that mirrors your actual code structure. You get run-level metadata, token counts per step, and the ability to click into any node in an agent's execution and see exactly what it received and returned.

• Best for: teams that have fully committed to the LangChain/LangGraph ecosystem and want tracing that understands their abstractions natively — chains, agents, retrievers, and output parsers show up as first-class objects, not generic spans.
• Strengths: deepest integration with LangChain-specific constructs, built-in dataset and evaluation tooling (you can turn traced runs directly into regression test datasets), prompt versioning and a prompt playground, and tight iteration loops if you're already writing LangChain code.
• Tradeoffs: it's a managed, hosted product first. There is a self-hosted enterprise tier, but it's aimed at larger organizations, not a weekend self-host on a spare VPS. And if you're not using LangChain — say you're calling the OpenAI or Anthropic SDK directly, or you're on a different framework — you lose a lot of the automatic instrumentation and end up doing manual span creation that feels like fighting the tool's natural shape.

The honest framing: LangSmith is not really "a tracing tool that also happens to support LangChain." It's a LangChain-native platform that happens to expose a generic tracing API. If you're all-in on LangChain, that's a feature. If you're not, it's friction you'll feel on day one and every day after.

Langfuse

Langfuse takes the opposite starting position: assume nothing about your framework, and give you a clean, generic API for traces, spans, generations, and scores that works identically whether you're using LangChain, raw HTTP calls to an inference API, a custom agent loop, or three different frameworks stitched together because that's what actually happened in your codebase over six months.

• Best for: teams that want framework-agnostic tracing, or teams that specifically want the option to self-host and own their observability data instead of routing every prompt and completion through a third party's servers.
• Strengths: open-source core, genuinely self-hostable (Docker Compose gets you a working instance in minutes, and there's a documented path to a production Kubernetes deployment), SDKs for Python and JS/TS that don't assume LangChain, plus first-class integrations for LangChain, LlamaIndex, and the OpenAI SDK when you do use them. It also ships with prompt management, dataset-based evals, and scoring — so it's not "just tracing," it covers a lot of the same ground as LangSmith without requiring the framework buy-in.
• Tradeoffs: because it doesn't automatically understand your framework's internals the way LangSmith understands LangChain, you sometimes write a bit more instrumentation code yourself to get the same granularity. Self-hosting also means you now own uptime, backups, and upgrades for another service — that's a real cost even if the software is free.

The framework-agnostic part matters more than it sounds like on paper. Most real production LLM systems are not clean LangChain applications by month six — they're a mix of raw SDK calls for the parts where the framework got in the way, a couple of custom retrieval steps, and maybe one legacy chain nobody wants to touch. Langfuse traces all of it under one roof without asking you to route everything back through a single framework's abstractions.

Helicone

Helicone takes a third approach entirely: instead of an SDK you wrap around your code, it's primarily a proxy. You point your API calls at Helicone's endpoint instead of directly at OpenAI, Anthropic, or whichever provider you're using, and it captures request/response pairs, latency, and cost as the traffic passes through. For a lot of teams, that's a one-line change — swap the base URL, add an auth header, done.

• Best for: teams that want cost and latency visibility fast, with minimal code changes, especially when the goal is "tell me what we're spending and where" rather than "let me inspect the full reasoning tree of a five-step agent."
• Strengths: genuinely the lowest integration cost of the three — proxy-based setup can be live in minutes. Strong cost dashboards, request-level logs, caching support, and rate-limiting built into the same proxy layer. It also has an open-source version if you want to self-host the proxy.
• Tradeoffs: the proxy model is shallower by design. It sees individual LLM calls very well, but it doesn't automatically understand the relationship between five calls that make up one logical agent turn unless you do extra work to tag and group them. If you need to visually drill into a nested trace tree — this retrieval fed this prompt, which triggered this tool call, which fed this second prompt — Helicone can be made to do it, but it's not the primary mental model the way it is for LangSmith or Langfuse.

Where Helicone genuinely shines is the "I just need to know what this is costing me and whether it's slow" use case. If you have a single-call summarization endpoint, a classification pipeline, or any workload where each request is basically one LLM call, Helicone gets you a production-grade cost and latency dashboard with the least amount of engineering effort of any option here.

Self-hosting vs managed: what you're actually trading off

This is the decision that quietly determines a lot of the rest. Managed hosting (LangSmith's default mode, Helicone's default mode) means someone else runs the ingestion pipeline, the database, the dashboard, and the scaling. You sign up, get an API key, and start sending traces. The cost is that your prompts and completions — which frequently contain user data, business logic, and sometimes literally your system prompts, which are IP — now live on a third party's infrastructure. For some teams that's a complete non-issue. For teams in regulated industries, or teams that consider their prompts competitively sensitive, it's disqualifying.

Self-hosting, which is Langfuse's headline strength and available for Helicone's proxy as well, keeps all of that inside your own infrastructure. You run the containers, you own the Postgres/Clickhouse instance underneath, you control retention and access. The tradeoff is operational: you're now responsible for patching, scaling the ingestion path when trace volume spikes, and upgrading across versions. A two-person startup that self-hosts Langfuse and then ignores it for four months will eventually hit a disk space problem or a version drift problem at the worst possible time.

A reasonable rule of thumb: if you have no dedicated infra person and your data isn't especially sensitive, start managed with any of the three and revisit later. If you have compliance requirements, a strong preference for data ownership, or you're already running your own Kubernetes cluster anyway, self-hosted Langfuse removes an entire category of "is this data leaving our network" conversations before they start.

Framework lock-in considerations

Framework lock-in is easy to underestimate early and expensive to fix late. LangSmith's tracing quality is directly proportional to how much of your application is expressed in LangChain/LangGraph primitives. That's a fair trade if you're confident you'll stay on LangChain for the life of the project. But teams that adopted LangChain early, then found themselves ripping out a chain here and a chain there in favor of direct SDK calls for performance or clarity, often find their LangSmith traces get patchier over time — some parts beautifully instrumented, others manually wrapped and less informative.

Langfuse's bet is that frameworks will keep changing under you, but the shape of "trace in, generation out, score attached" doesn't. Because it instruments at the level of the actual model call rather than the framework's internal objects, switching from LangChain to LlamaIndex to a hand-rolled agent loop doesn't mean re-architecting your observability — you keep the same trace/span/generation vocabulary throughout.

Helicone mostly sidesteps the framework question because it operates below the framework layer entirely, at the HTTP boundary. It doesn't care if you called the API via LangChain, LlamaIndex, or a bare requests.post — it sees the request either way. That's exactly why it's shallow on trace structure: it never sees your code's logical grouping, only the wire traffic.

Cost tracking granularity

All three will tell you how many tokens you used and roughly what it cost. The differences show up when you ask a harder question, like "which customer's usage of our support bot is driving 40% of our OpenAI bill this month" or "did switching from one model to a cheaper one on the summarization step actually save money after accounting for the retries it caused."

• LangSmith: cost and token metrics are attached to each traced run, so you can filter and aggregate by any metadata you tag onto the run (user ID, session, feature flag). Good granularity as long as you're tagging consistently, which is on you to set up.
• Langfuse: similarly tag-driven, with the advantage that because it's framework-agnostic you can attach the same metadata scheme across every part of a mixed-framework system, and its dashboards are built specifically around cost-per-trace, cost-per-user, and cost-per-feature breakdowns — this is one of its strongest areas precisely because "cost observability across a heterogeneous stack" is the problem it was designed for.
• Helicone: this is arguably its best-in-class feature. Because every request passes through the proxy, cost tracking is automatic and comprehensive by default — you don't have to remember to instrument anything for it to show up. Per-user cost attribution, caching hit rates that directly reduce spend, and rate-limit dashboards come essentially for free. If cost visibility is your primary or even only goal, Helicone gets you there with the least setup of the three.

The practical distinction: Helicone gives you excellent cost visibility with almost no effort, because that's the one thing the proxy sees clearly no matter what. LangSmith and Langfuse give you cost data that's exactly as good as your tagging discipline, but tied to the richer trace context, so you can connect a cost spike to the specific chain, agent step, or prompt version that caused it — not just the dollar figure.

When a lightweight proxy is enough vs when you need deep trace trees

This is the question to actually ask before picking a tool, more than any feature checklist.

A lightweight proxy approach (Helicone) is enough when:

• Your application is mostly single-call: one prompt in, one completion out, maybe with retries.
• Your primary concern is spend — knowing what you're paying, per model, per user, per day — and catching latency regressions.
• You want caching and rate limiting handled at the same layer as observability, without adding another SDK dependency to your codebase.
• You're early stage and want the fastest possible path to "we have visibility into production," measured in minutes, not days.

You need deep trace trees (LangSmith or Langfuse) when:

• Your application is agentic or multi-step: a request triggers retrieval, then a tool-calling decision, then possibly a second or third LLM call, and you need to see the whole tree to debug a bad output.
• You're building evaluation pipelines where you need to replay full traces against new prompt versions or new models, not just look at aggregate metrics.
• Different steps in your pipeline use different prompts or models, and "which step caused this regression" is a question you'll be asking weekly, not once a quarter.
• You want structured feedback scores (thumbs up/down, LLM-graded quality scores, human annotation) attached at the level of individual trace steps, not just the overall interaction.

A pattern worth naming: it's common to start with something proxy-simple like Helicone because the app starts as a single-call wrapper, and then outgrow it the moment you add your first agent loop or RAG pipeline with more than one hop. That's not a failure of Helicone — it did exactly what it was built for. It's a sign the application's shape changed and the observability tool needs to change with it.

A framework-agnostic tracing setup, conceptually

Here's roughly what a Langfuse-style, framework-agnostic tracing setup looks like conceptually — the point being that this same shape works whether the underlying call is a raw API request or buried inside a framework's internals:

from langfuse import Langfuse

langfuse = Langfuse(
    public_key="pk-...",
    secret_key="sk-...",
    host="https://your-self-hosted-instance.example.com",  # or the managed cloud host
)

# Start a trace for one end-to-end user interaction
trace = langfuse.trace(
    name="support-agent-turn",
    user_id="user_123",
    metadata={"session_id": "sess_456", "feature": "support_bot"},
)

# A retrieval step, modeled as its own span
retrieval_span = trace.span(name="retrieve-context")
retrieved_docs = my_retriever.search(query)
retrieval_span.end(output={"num_docs": len(retrieved_docs)})

# The actual LLM call, modeled as a "generation" with cost-relevant fields
generation = trace.generation(
    name="answer-synthesis",
    model="gpt-4.1",
    input=prompt,
    metadata={"retrieved_doc_ids": [d.id for d in retrieved_docs]},
)

response = call_llm(prompt)  # this could be a raw API call OR a framework call underneath

generation.end(
    output=response.text,
    usage={
        "input_tokens": response.usage.input_tokens,
        "output_tokens": response.usage.output_tokens,
    },
)

# Attach a quality score later, e.g. from a user thumbs-up/down or an automated grader
trace.score(name="user_feedback", value=1, comment="thumbs up")

Notice what's absent: no LangChain callback handler, no framework-specific hook. The trace / span / generation vocabulary is generic enough to wrap around anything that takes an input and produces an output, which is exactly why this pattern travels well across a codebase that isn't cleanly one framework end to end. You could wrap this same structure around a LangChain .invoke() call, a raw HTTP request, or a call into a completely different SDK next year, and your historical traces would still line up under the same schema.

Picking one, practically

If you're fully committed to LangChain or LangGraph and expect to stay that way, LangSmith gives you the least friction and the tightest integration with the abstractions you're already using — the trace tree essentially writes itself. If you want framework independence, the option to self-host, and a tool that treats "raw API calls next to LangChain calls next to a custom agent loop" as a normal, unremarkable thing to trace, Langfuse is the more durable choice, especially as systems grow past their first framework. If what you actually need right now is "tell me what we're spending and whether latency is creeping up," with the least engineering effort of the three, Helicone's proxy model gets you there fastest — and you can always add deeper tracing later without throwing away the cost dashboards you already built.

None of these tools replace the harder work that comes after you have the traces: deciding whether an output was actually good. That's where evaluation enters the picture, and increasingly, teams handle it by using LLM-as-a-Judge — a second model call, run over your traced inputs and outputs, that scores quality against a rubric so you're not stuck manually reading every log line by hand. Observability gets you the trace. LLM-as-a-Judge is what turns that trace into a signal you can act on.