LangSmith Trace Retention: Managing Storage Costs Over Time

Every LLM application that ships to production eventually hits the same wall. You instrument your chains with LangSmith, the traces start flowing, and for the first few weeks everything feels magical. You can see every prompt, every token count, every latency spike, every tool call your agent made. Then the monthly bill arrives, and the number is bigger than you expected. Not because LangSmith is expensive, but because you never thought about what happens to a trace after you stop looking at it. Traces do not evaporate. They sit in storage, quietly accumulating, and unless you have a retention strategy they will keep accumulating until the cost curve bends in a direction you do not like.

This is the part of observability nobody puts in the demo. Getting data in is easy. Deciding how long to keep it, at what fidelity, and for which slices of traffic is the actual engineering problem. In this article we are going to walk through how LangSmith trace retention works, why costs grow the way they do, and the concrete tactics you can use to keep spend flat even as your traffic multiplies. None of this requires ripping out your instrumentation. Most of it is a matter of tagging correctly, sampling deliberately, and understanding the difference between the traces you need forever and the traces you needed for about six hours.

Why Trace Volume Grows Faster Than You Think

The first thing to internalize is that trace count does not grow linearly with users. It grows with the product of users, requests per user, and spans per request. A single agent invocation is not one trace. It can be a parent run with a dozen child runs underneath it: the retriever call, the reranker, three separate LLM calls for a multi-step reasoning loop, a couple of tool executions, and a final synthesis step. Each of those is a run that LangSmith records, with inputs, outputs, metadata, and timing.

So when your product team says traffic tripled, your trace volume did not triple. If the new traffic skews toward the agentic feature that fans out into ten spans instead of the simple completion endpoint that produces one, your trace volume might have grown eightfold. The mental model that gets people into trouble is thinking of a trace as a request. A trace is a tree, and trees have leaves.

There is also a payload dimension. A trace that carries a 50-token prompt is cheap. A trace that carries a retrieved context window of 40,000 tokens plus the full document chunks plus the model's 4,000-token response is a heavy object. Retrieval-augmented generation applications are notorious here because the whole point of RAG is stuffing large contexts into prompts, and every one of those large contexts gets serialized into the trace. Two applications with identical request counts can have wildly different storage footprints depending on how fat their payloads are.

The practical takeaway is that you should measure your trace volume in terms of runs and bytes, not user-facing requests. Once you start thinking in runs and bytes, retention decisions become obvious.

How LangSmith Retention Actually Works

LangSmith organizes trace data around retention tiers, and understanding these tiers is the foundation of cost control. At a high level, traces are ingested and then held according to a retention policy. There is a distinction worth burning into your memory: some traces are kept in full fidelity for a longer window because you flagged them as important, and the rest are kept for a shorter default window before they are aged out.

The key lever LangSmith gives you is the ability to control which traces get extended retention and which get the shorter default. This is not an all-or-nothing switch. You decide, per trace or per project, what deserves to stick around. A trace tied to a customer support escalation, a trace that triggered an error, or a trace you want to use as an evaluation example deserves long retention. A trace from a healthy, boring, successful request that returned in 200 milliseconds and made nobody unhappy probably does not need to live for months.

The cost model follows directly from this. You pay based on how many traces you ingest and how long you retain them at what fidelity. Extended retention costs more than base retention because it holds the full object for longer. This means the single most powerful thing you can do is make a deliberate decision about the retention tier of each trace, rather than letting everything default into the same bucket.

Here is a simple way to think about the categories of traces you are dealing with:

• Traces you will inspect within hours and never touch again
• Traces that are interesting only because something went wrong
• Traces you want to keep permanently as golden examples for evaluation
• Traces that exist purely for aggregate metrics and do not need per-run detail

Each of these maps to a different retention strategy, and mixing them together is exactly why bills balloon.

Sampling: The Highest-Leverage Cost Control

If you take one idea away from this article, make it this: you almost certainly do not need to trace 100 percent of production traffic at full fidelity. Sampling is the highest-leverage cost control available, and it is dramatically underused because it feels scary. It feels like you are throwing away data you might need. In practice, you are throwing away the ninety-ninth redundant example of a request that behaves exactly like the previous ninety-eight.

The trick is to sample intelligently rather than uniformly. Uniform sampling, keeping one in every ten traces regardless of content, is better than nothing but blunt. Smart sampling keeps everything that is anomalous and samples down everything that is routine. You want to keep all the errors, all the slow requests, all the low-confidence outputs, and a small representative fraction of the happy path.

Here is a pattern for conditional tracing based on the outcome of a request. The idea is to decide at runtime whether a given trace is worth keeping at full fidelity:

import random
from langsmith import traceable
from langsmith.run_helpers import get_current_run_tree

# Keep a small fraction of successful, fast requests.
HAPPY_PATH_SAMPLE_RATE = 0.05

def should_retain(latency_ms: float, had_error: bool, confidence: float) -> bool:
    # Always keep anything that looks abnormal.
    if had_error:
        return True
    if latency_ms > 3000:
        return True
    if confidence < 0.4:
        return True
    # Otherwise keep only a sample of the boring successes.
    return random.random() < HAPPY_PATH_SAMPLE_RATE

@traceable(run_type="chain")
def handle_request(query: str):
    result, latency_ms, confidence, had_error = run_pipeline(query)

    run = get_current_run_tree()
    if run is not None:
        keep = should_retain(latency_ms, had_error, confidence)
        # Tag the trace so retention policy can act on it later.
        run.add_metadata({"retain": keep, "sampled": not keep})

    return result

Notice what this does. It never blinds you to problems, because every error and every slow or low-confidence request is retained. It only reduces the volume of the traces that all look the same. In a mature system this can cut ingested trace volume by an order of magnitude while preserving essentially all of the debugging value.

Tagging and Metadata as a Retention Strategy

Sampling decides whether a trace exists. Tagging decides how a trace that exists gets treated over time. These are complementary, and together they form the backbone of a retention policy. The goal is to attach enough structured metadata at ingestion time that later you, or an automated process, can make retention decisions without having to re-inspect the trace content.

Good metadata to attach includes the environment, the feature or route, the model used, a user or tenant identifier bucketed for privacy, and a category flag that describes why the trace matters. When you tag consistently, you unlock the ability to answer questions like "how much of my storage is being consumed by the experimental beta feature that only 2 percent of users touch" and then act on the answer.

Here is a compact example of attaching structured metadata and tags at trace creation:

from langsmith import traceable
from langsmith.run_helpers import get_current_run_tree

@traceable(run_type="chain", tags=["prod", "rag-search"])
def answer_question(query: str, tenant_id: str, model: str):
    run = get_current_run_tree()
    if run is not None:
        run.add_metadata({
            "environment": "production",
            "feature": "rag-search",
            "model": model,
            "tenant_bucket": hash(tenant_id) % 100,
            "trace_category": "user_facing",
        })
    return run_rag(query, model)

The payoff comes weeks later. When you sit down to review cost, structured tags let you slice storage by feature and environment. Almost every team that does this exercise for the first time discovers a surprise: some staging or development project that nobody remembered to turn off has been quietly ingesting traces at full fidelity for a month. Tags make that visible. Without tags, all your traces look identical in aggregate and you cannot reason about where the cost is coming from.

A discipline worth adopting is treating your development and staging environments as first-class retention concerns. Non-production traffic is often noisier and higher-volume than production because it includes load tests, integration test suites, and developers hammering the same endpoint fifty times while debugging. That traffic has almost no long-term value. It should be sampled aggressively and retained briefly.

Controlling Payload Size Before It Reaches Storage

Even with perfect sampling and tagging, individual traces can be bloated. The single biggest offender is oversized inputs and outputs, and the biggest single offender within that is retrieved context in RAG systems. You do not always need to store the entire 40,000-token context blob inside the trace. Often what you need is a reference to which documents were retrieved and maybe a truncated preview, not the full text of every chunk.

Trimming payloads before they are recorded is a legitimate and effective cost tactic. The principle is to store what you would actually look at during debugging and drop or summarize the rest. If you would never scroll through the full concatenated context during an investigation, do not pay to store it.

Here is a helper that truncates large fields before they become part of a trace, while preserving enough to be useful:

def trim_for_trace(value, max_chars: int = 2000):
    text = value if isinstance(value, str) else str(value)
    if len(text) <= max_chars:
        return text
    head = text[: max_chars // 2]
    tail = text[-max_chars // 2 :]
    return f"{head}\n...[trimmed {len(text) - max_chars} chars]...\n{tail}"

def build_trace_inputs(query: str, retrieved_docs: list[str]):
    return {
        "query": query,
        "num_docs": len(retrieved_docs),
        "doc_ids": [d[:40] for d in retrieved_docs][:20],
        "context_preview": trim_for_trace("\n".join(retrieved_docs)),
    }

The behavior here matters. You keep the query in full because it is small and always relevant. You keep the count and identifiers of retrieved documents so you can reconstruct what happened. You keep a trimmed preview of the context so you can eyeball whether retrieval pulled the right material. What you drop is the tens of thousands of redundant tokens that you would never actually read. The debugging experience is nearly identical, and the storage footprint per trace can shrink by 90 percent or more on heavy RAG workloads.

There is a caveat to state plainly. If you are using traces as evaluation datasets and you need the full context to reproduce a run, do not trim those. This is exactly why the retention-tier decision and the trimming decision should be linked. Golden traces destined for evaluation keep full payloads and long retention. Routine traces get trimmed payloads and short retention. The two categories should never be treated the same way.

Separating Long-Term Evaluation Data From Ephemeral Debug Data

One of the most useful mental shifts is to stop thinking of your trace store as a single undifferentiated pool and start thinking of it as two very different datasets that happen to share a pipeline. The first dataset is ephemeral operational data: the stream of traces you use to debug what is happening right now and to compute live metrics. This data has a short useful life. A trace from last Tuesday that succeeded is worthless to you today.

The second dataset is durable evaluation data: the curated set of examples you have deliberately promoted into datasets because they represent important cases, hard cases, or regression risks. This data is precious and should be retained indefinitely, but it is also tiny. A high-quality evaluation set might be a few hundred or a few thousand examples, not a few million.

The cost mistake is retaining the operational firehose as if it were the evaluation set. You end up paying long-retention prices for millions of traces you will never open, in order to preserve a few thousand you actually care about. The fix is to explicitly promote the traces you want to keep into managed datasets and let everything else age out on a short schedule.

Here is the shape of that promotion, pulling a specific run into a persistent dataset via the LangSmith client:

from langsmith import Client

client = Client()

def promote_run_to_dataset(run_id: str, dataset_name: str):
    run = client.read_run(run_id)

    datasets = list(client.list_datasets(dataset_name=dataset_name))
    if datasets:
        dataset = datasets[0]
    else:
        dataset = client.create_dataset(
            dataset_name=dataset_name,
            description="Curated golden examples for regression evaluation",
        )

    client.create_example(
        inputs=run.inputs,
        outputs=run.outputs,
        dataset_id=dataset.id,
        metadata={"source_run": str(run_id)},
    )
    return dataset.id

Once a run is promoted, its long-term value is captured in the dataset, and the original trace no longer needs extended retention. You have decoupled the thing you care about from the storage tier of the raw trace. This is the single cleanest way to reconcile the tension between wanting to keep important examples forever and not wanting to pay to keep everything forever.

Building a Retention Review Into Your Operational Cadence

Retention is not a set-it-and-forget-it configuration. Traffic patterns shift, new features ship, and a sampling rate that was correct three months ago may be wrong today. The teams that keep observability spend under control treat retention as a recurring operational review, the same way they review error budgets or on-call load. It does not need to be heavy. A short monthly pass is usually enough.

A practical review checklist looks like this:

1. Pull the current trace volume broken down by project, environment, and feature tag.
2. Identify the top three consumers of storage and ask whether each one earns its cost.
3. Check for any non-production project that is ingesting more than expected and tighten its sampling.
4. Confirm that error and high-latency traces are still being retained at full fidelity.
5. Review which datasets have grown and prune stale evaluation examples that no longer reflect current behavior.
6. Adjust the happy-path sample rate up or down based on how much debugging you actually did from sampled traces.

The fifth point deserves emphasis because it runs against instinct. Evaluation datasets can also become a cost and a liability if they grow without pruning. An example that reflects a prompt you deprecated six months ago is not helping your evaluations, it is polluting them. Curating down is as important as curating in.

The sixth point is the feedback loop that keeps sampling honest. If over the last month you never once needed to open a sampled happy-path trace, your sample rate is probably too high and you can lower it. If you found yourself wishing you had a trace that got sampled away, nudge it up. Sampling rates should be tuned by evidence, not set once and forgotten.

Common Retention Mistakes and How to Avoid Them

A few failure modes show up again and again, and knowing them in advance saves real money. The first is the forgotten environment. Someone spins up a staging project, points a load test at it, and forgets it exists. Weeks later it is one of the largest line items on the bill. The defense is consistent environment tagging and the monthly review that surfaces unexpected volume.

The second mistake is tracing sensitive data at full fidelity and then being forced to retain it, which is both a cost problem and a compliance problem. If your traces contain personal data, retention length is no longer purely an economic decision, it is a governance one. Trimming and redacting payloads before they are recorded solves both concerns at once, which is a nice property. The same technique that shrinks storage also shrinks your exposure surface.

The third mistake is treating sampling as a threat to observability rather than a tool for it. Engineers resist sampling because they imagine the one time they will need the trace that got dropped. But smart sampling never drops the interesting traces. It only drops redundant copies of normal behavior. Once you frame it as keeping all the signal and discarding the duplicate noise, the resistance usually dissolves.

The fourth mistake is optimizing storage without measuring it first. Do not guess at where your costs come from. Tag everything, then look at the actual breakdown before you change anything. Almost every team that measures discovers the cost distribution is lopsided, with one or two features or environments responsible for the large majority of the volume. Optimizing those first gives you most of the savings for a fraction of the effort, and it prevents you from over-engineering sampling on traffic that was never the problem.

Bringing It All Together

Managing LangSmith trace retention is not about tracing less. It is about tracing deliberately. The applications that keep observability cheap at scale are not the ones with the fewest traces, they are the ones that decided, on purpose, what each trace is for. Error traces and slow traces are kept at full fidelity because they earn their cost in debugging value. Golden examples are promoted into durable datasets and kept forever because they are small and precious. The vast redundant middle, the endless stream of successful requests that all look alike, is sampled down and aged out quickly because keeping the hundred-thousandth copy of normal teaches you nothing.

Put the pieces together and you have a system that stays flat in cost even as traffic grows. Sample intelligently so volume tracks signal rather than raw request count. Tag consistently so you can always answer where your storage is going. Trim payloads so individual traces stay lean. Separate ephemeral operational data from durable evaluation data so each lives on the retention schedule it deserves. Review the whole thing on a monthly cadence so it adapts as your product changes. None of these steps is difficult on its own, and none requires you to give up the visibility that made you adopt tracing in the first place.

The reward is an observability practice you can actually afford to run in production indefinitely, one that gets more useful over time rather than more expensive. That is the whole game: full visibility into what matters, near-zero spend on what does not, and a cost curve that stays boring no matter how many users show up.

If you want to go deeper on instrumenting, sampling, tagging, and building evaluation datasets the right way from day one, the LangSmith Tutorial course on teachyou.ai walks through the entire workflow hands-on, from your first traced chain to a production-grade retention strategy you can defend to your finance team. It is the fastest way to turn these principles into a setup that runs itself.