LangSmith Cost Tracking: Understanding Your LLM Spend

Your LLM application works beautifully in the demo, ships to production, and then the first invoice arrives. Suddenly everyone wants to know the same thing: where is the money actually going? Which feature, which prompt, which user, which retry loop? Most teams discover at this exact moment that they have no answer, because token spend is invisible by default. It hides inside API calls, spread across dozens of chains, agents, and retries, and the provider's billing page only tells you the total damage, not the cause. This is the problem LangSmith cost tracking solves. Because LangSmith already records every LLM call as part of a trace, it is perfectly positioned to attach a dollar value to each one, roll those values up into traces, projects, and dashboards, and let you slice spend by model, feature, or customer. In this guide we will walk through how LangSmith computes cost, how to set it up correctly for both mainstream and custom models, how to attribute spend to the people and features that generate it, and how to use that visibility to actually reduce your bill.

Why LLM Cost Tracking Is Harder Than It Looks

Traditional infrastructure cost is relatively predictable: you provision servers, you pay for servers. LLM cost is usage-based at an extremely fine granularity, and that changes everything.

First, cost is proportional to tokens, not requests. Two calls to the same endpoint can differ in cost by a factor of a hundred depending on how much context you stuff into the prompt and how long the completion runs. A single verbose system prompt duplicated across every request can quietly dominate your bill.

Second, modern LLM applications are not single calls. A RAG pipeline might embed a query, call a reranker, and then call a chat model. An agent might loop through five or ten tool-calling iterations before producing a final answer. From the outside, that is "one user request." From a billing perspective, it is a dozen separate charges. If you only measure at the request boundary, you cannot see that the third tool-call iteration is the expensive one.

Third, pricing itself is asymmetric and volatile. Input tokens and output tokens are priced differently, often by a factor of three to five. Cached input tokens are priced differently again. Providers revise prices, launch new models, and deprecate old ones. Any hardcoded cost calculation you write yourself starts rotting the day you ship it.

Finally, cost questions are almost always attribution questions. "We spent a lot this month" is useless. "The summarization feature spent three times more than search, and one enterprise tenant accounts for most of it" is actionable. Attribution requires metadata attached to every call, and that is precisely what a tracing system provides.

LangSmith cost tracking addresses all four problems with the same underlying mechanism: every LLM invocation becomes a run inside a trace, every run carries token counts, and LangSmith multiplies those counts against a pricing table to produce a cost that rolls up through the trace tree.

How LangSmith Calculates Cost Under the Hood

The model is simple and worth internalizing, because once you understand it, every debugging session gets faster.

When an LLM call is traced, LangSmith records a run of type llm. That run includes token usage: input tokens, output tokens, and total tokens, plus finer-grained details like cache reads when the provider reports them. For popular SDKs and LangChain integrations, this usage metadata is extracted automatically from the provider's response object, so you get token counts without writing any extra code.

LangSmith then consults its model pricing map. This is a table, visible and editable in your workspace settings, that maps a model name pattern to a price per million input tokens and a price per million output tokens. LangSmith ships with entries for the widely used models from major providers, and it matches your run against the table using the model name recorded on the run. The match supports pattern matching, which matters in practice because providers version their model names, and you want one pricing rule to cover a family of dated releases rather than editing the table every time a snapshot changes.

Cost is then computed per run:

• Input cost is input tokens multiplied by the input price.
• Output cost is output tokens multiplied by the output price.
• Total run cost is the sum, adjusted for any special token categories such as cached prompt tokens where a separate rate applies.

The crucial part is aggregation. Costs roll up from child runs to parent runs, so a trace representing one user request shows the combined cost of every LLM call inside it. Project-level stats aggregate across traces. This means you can answer questions at any altitude: what did this specific agent loop cost, what does an average request to this feature cost, and what did the whole project cost this week.

One honest caveat: LangSmith computes an estimate based on token counts and the pricing table. It is not a reconciliation of your provider invoice. Rounding, provider-side billing quirks, and calls that bypass tracing will produce small discrepancies. Treat LangSmith cost tracking as your operational lens for decisions and debugging, and treat the provider invoice as the accounting source of truth. In practice the two track each other closely if your tracing coverage is complete.

Setting Up Cost Tracking in Minutes

If you are already tracing with LangSmith, cost tracking largely works out of the box for supported models. The setup is the standard tracing setup.

Set the environment variables:

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="your-api-key"
export LANGSMITH_PROJECT="my-production-app"

Then make sure your LLM calls actually flow through instrumentation. With LangChain, tracing is automatic once the variables are set. If you call the OpenAI SDK directly, wrap the client:

from openai import OpenAI
from langsmith.wrappers import wrap_openai
from langsmith import traceable

client = wrap_openai(OpenAI())

@traceable(name="summarize_ticket")
def summarize_ticket(ticket_text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Summarize the support ticket in two sentences."},
            {"role": "user", "content": ticket_text},
        ],
    )
    return response.choices[0].message.content

The wrap_openai wrapper captures the request, the response, and the usage block that the API returns, including prompt and completion token counts. Because the model name is recorded on the run and it matches an entry in the default pricing map, LangSmith computes the cost automatically. Open the trace in the UI and you will see token counts and a dollar figure on the LLM run, and the parent run shows the aggregate.

Anthropic users get the same experience with the corresponding wrapper, and LangChain users get it for any integration that populates usage metadata on the model output. The general rule: if the run has token counts and the model name matches the pricing map, LangSmith cost tracking is on. No separate billing SDK, no extra service.

Two things are worth verifying early. First, check that your token counts are non-zero in the trace view; streaming configurations in some SDKs need the option that includes usage in the final chunk, otherwise the response never carries token numbers for LangSmith to read. Second, check the model name recorded on the run matches how the pricing map expects it, especially if you route through a proxy or gateway that rewrites model identifiers.

The Model Pricing Map: Your Source of Truth for Rates

The pricing map deserves attention because it is the one place where dollars enter the system. You will find it in the LangSmith settings for your workspace.

Each entry contains a match pattern for the model name, an optional provider, a price per million prompt tokens, and a price per million completion tokens. Default entries cover the mainstream hosted models. You should still review them for the models you use, for three reasons.

• Prices change. When a provider cuts prices or you migrate to a new snapshot, the map must reflect it or your estimates drift.
• You may use models the defaults do not cover: fine-tuned variants, open-weight models served through an inference provider, or internal deployments.
• You may want internal pricing. Some platform teams deliberately set rates above raw provider cost to account for infrastructure overhead, effectively creating an internal chargeback rate.

Adding a custom entry is a form fill: give it a pattern such as my-finetuned-model.*, set the two rates, and every past-forward run whose model name matches begins receiving cost estimates. Pattern matching means a single rule like gpt-4o-mini.* covers dated snapshots without maintenance.

A practical tip for self-hosted models: even though your marginal token cost is not billed by anyone, assigning an estimated rate based on your GPU costs turns LangSmith into a unified spend view across hosted and self-hosted inference. Without it, self-hosted calls show up as zero cost and silently skew every comparison you make between models.

Also note the precedence rule: newer, more specific entries you create take priority over the built-in defaults, so you can override a default rate without deleting it.

Tracking Costs for Custom and Wrapped Models

The automatic path covers wrapped SDK clients and LangChain integrations. But real systems often have calls that do not fit that mold: a REST call to an internal inference server, a vendor SDK without a wrapper, or a batch pipeline that processes responses offline. For these, you tell LangSmith the numbers yourself.

The contract is straightforward: create a run of type llm, record the model name where the pricing map can find it, and attach usage metadata with token counts. With the Python SDK:

from langsmith import traceable

@traceable(
    run_type="llm",
    metadata={"ls_provider": "my_provider", "ls_model_name": "my-custom-model-v2"},
)
def call_custom_model(messages: list) -> dict:
    # Call your internal inference endpoint however you like
    result = my_inference_client.generate(messages)

    return {
        "choices": [
            {"message": {"role": "assistant", "content": result.text}}
        ],
        "usage_metadata": {
            "input_tokens": result.prompt_tokens,
            "output_tokens": result.completion_tokens,
            "total_tokens": result.prompt_tokens + result.completion_tokens,
        },
    }

Three details make this work. The run_type="llm" marks the run as an LLM invocation so LangSmith looks for usage on it. The ls_model_name metadata key is how LangSmith identifies which pricing map entry applies. And the usage_metadata block in the return value carries the token counts that get multiplied against the rates. Pair this with a pricing map entry matching my-custom-model-v2 and the run shows a cost like any OpenAI call would.

If your provider reports token usage with different semantics, or you want to account for cached tokens, the usage metadata supports finer-grained detail fields, and the pricing map can price those categories separately. The same mechanism also lets you correct usage after the fact: if token counts only become known later (for example, in a batch API workflow), you can update the run with the final usage so the cost reflects reality.

The takeaway: there is no model LangSmith cannot track cost for. If you can count tokens, you can price them.

Reading the Numbers: Traces, Dashboards, and Filters

Collecting cost data is step one. The value comes from interrogating it, and LangSmith gives you several altitudes to work at.

At the trace level, open any trace and look at the run tree. Every LLM run displays its token counts and cost, and every parent aggregates its children. This is where you diagnose individual expensive requests. A pattern you will see constantly in agentic systems: the final answer generation is cheap, but the trace shows six tool-selection calls before it, each re-sending the full conversation history, and together they dominate the cost. You cannot see this from a billing dashboard; you can see it in ten seconds in a trace tree.

At the project level, the monitoring section charts cost, token counts, latency, and trace volume over time. This is where trend questions live: is cost per trace creeping upward, did the deploy on Tuesday change our token profile, is total spend growing faster than traffic. A rising cost-per-trace with flat traffic is your earliest warning that a prompt change or a retry bug is inflating context sizes.

Filtering is where LangSmith cost tracking becomes genuinely powerful. The trace filter syntax lets you slice by time range, tag, metadata key, model, feedback score, and more, and the aggregates recompute for the filtered set. Some queries worth having in your back pocket:

• Sort traces by total cost descending to find your most expensive individual requests and read them.
• Filter by a metadata key like feature:summarize and compare cost statistics against feature:search.
• Filter by error status to measure how much money failed requests burn, since a request that fails after four LLM calls still costs you four LLM calls.
• Compare cost distributions before and after a prompt change by filtering on the metadata you attach for prompt versions.

Custom dashboards let you pin these slices as charts, so cost by feature or cost by model becomes something the whole team glances at rather than something one engineer queries when worried. Reviewing that dashboard weekly, the same way you review error rates, is the habit that separates teams who control LLM spend from teams who get surprised by it.

Attributing Spend to Users, Features, and Tenants

The single highest-leverage practice in LLM cost management is disciplined metadata. Cost numbers without attribution tell you that you are spending; cost numbers with attribution tell you why.

LangSmith lets you attach arbitrary metadata and tags to traces, and every one of those keys becomes a filterable dimension for cost analysis. The pattern looks like this:

from langsmith import traceable

@traceable(name="handle_user_request")
def handle_user_request(user_id: str, tenant: str, query: str):
    return run_pipeline(
        query,
        langsmith_extra={
            "metadata": {
                "user_id": user_id,
                "tenant": tenant,
                "feature": "document_qa",
                "prompt_version": "v13",
            }
        },
    )

With that in place, an entire class of business questions becomes a filter expression:

1. Per-customer cost: filter by tenant and read the aggregate. Essential for anyone selling usage-based or seat-based plans, because it tells you which accounts are profitable and whether your pricing covers your heaviest users.
2. Per-feature cost: filter by feature to learn what each capability costs to operate. This is how you discover that the nice-to-have auto-title generator costs more than the core search experience.
3. Per-version cost: attach a prompt_version or experiment identifier and you can quantify the cost impact of every prompt iteration, not just its quality impact.
4. Per-user anomalies: sorting by user makes abuse visible. A single user hammering your most expensive endpoint shows up immediately as an outlier.

A useful convention is to decide on your attribution keys early and enforce them at the entry point of your application, not sprinkled ad hoc through the codebase. If every trace is guaranteed to carry tenant, feature, and prompt_version, then every future cost question is answerable retroactively. Metadata you did not attach is analysis you cannot do.

Turning Visibility into Savings

Observability does not lower your bill by itself; the decisions it enables do. Here are the reductions that LangSmith cost tracking most commonly surfaces, roughly in order of how often teams find them.

Oversized context is the classic one. Trace trees make prompt bloat visually obvious: an input token count in the tens of thousands feeding a two-sentence answer. Common culprits are unbounded chat history, retrieval steps that stuff ten documents into context when two would do, and verbose system prompts repeated on every call. Fixes are mundane and effective: trim history with summarization, tighten retrieval k, and shorten instructions. Because cost is per token, a thirty percent context reduction is a thirty percent cost reduction on that call, and you can verify the effect in the dashboard the same day.

Model routing is the second. Once you can compare cost per call across models side by side with quality feedback, you often find entire call sites where a small model performs indistinguishably from the flagship one. Classification steps, routing decisions, title generation, and formatting passes rarely need the most capable model. Run the comparison as a LangSmith experiment, confirm quality holds, and move the traffic.

Agent loop discipline is the third. Traces expose how many iterations your agents actually take and what each iteration costs, since every loop typically re-sends the growing conversation. Capping iterations, summarizing intermediate state, and returning early on confident answers all show up directly as lower cost per trace.

Caching is the fourth. Provider-side prompt caching discounts repeated prefix tokens, and LangSmith records cache-read token counts so you can see whether your caching is actually engaging. If your traces show zero cached tokens despite a large shared system prompt, your prompt structure is probably breaking the cache prefix, and fixing the ordering is nearly free money. Application-level caching of full responses for repeated queries is visible too, as a drop in trace volume for the cached path.

Retry and error hygiene comes last but bites hardest when ignored. Filtered views of failed traces reveal how much you pay for requests that never deliver value. Aggressive retry policies on top of long prompts multiply waste; tightening backoff and failing fast on validation errors converts directly into savings.

The meta-point: every one of these optimizations existed in your system before you could see it. Cost tracking does not create the savings, it makes them findable and provable.

Common Pitfalls and How to Avoid Them

A few failure modes account for most confusing LangSmith cost tracking experiences, and all of them are avoidable.

Zero-cost runs usually mean one of three things: the run has no token usage recorded, the model name did not match any pricing map entry, or the call bypassed instrumentation entirely. Check them in that order. Streaming without usage reporting enabled is the most frequent cause of missing tokens; a proxy rewriting model names is the most frequent cause of failed pricing matches.

Partial tracing coverage quietly undermines trust. If half your LLM calls are wrapped and half are raw SDK calls, your project totals reflect half your spend, and someone will eventually compare them to the invoice and conclude the tool is wrong. Make instrumentation a code-review requirement for any new LLM call site.

Stale pricing entries skew comparisons. Put a recurring reminder on the pricing map, especially after model migrations. An outdated rate on your highest-volume model distorts every routing decision you make from the data.

Missing attribution metadata is the regret that cannot be fixed retroactively. You cannot filter last month's traces by a metadata key you started attaching yesterday. Standardize your keys now, even if you do not yet have the dashboards to consume them.

Finally, remember what the numbers are: operational estimates for engineering decisions. Use them to find waste, compare models, and attribute spend. Reconcile invoices with your provider's billing console.

Where to Go from Here

LangSmith cost tracking turns the vaguest anxiety in LLM engineering, "how much is this costing us," into a set of precise, filterable, chartable answers. The mechanics are approachable: token counts flow in from traced runs, the pricing map converts them to dollars, costs roll up through trace trees into projects and dashboards, and metadata turns totals into attribution. The practices around it are what compound: full instrumentation coverage, a maintained pricing map, disciplined metadata at the entry point, and a weekly habit of reading the cost dashboard like you read your error rates. Teams that do this stop being surprised by invoices and start treating cost as just another quality metric they engineer against.

If you want to go deeper, cost tracking is only one pillar of what LangSmith offers alongside tracing, evaluation, prompt management, and production monitoring, and they are far more powerful used together. Our LangSmith Tutorial course on teachyou.ai walks through the full workflow hands-on: instrumenting a real application, building cost and latency dashboards, attributing spend per customer, and running evaluation-backed experiments that cut your bill without cutting quality. If your LLM app is heading to production, it is the fastest way to make sure your budget survives the trip.