LangSmith for Multi-Tenant Applications: Isolating Traces Per Customer

Why Multi-Tenancy Breaks Naive LLM Observability

You shipped an LLM feature. It worked in the demo, so you wired up LangSmith, saw the traces flow in, and moved on. Then you signed your tenth customer, and now every trace lands in one giant undifferentiated stream. Customer A's support-bot conversation sits three rows above Customer B's contract-analysis run, and when Customer A emails asking why their agent hallucinated a refund policy, you have no clean way to pull only their runs. You are grepping through a shared project by timestamp and praying.

This is the moment most teams realize that observability for a single-tenant prototype and observability for a real SaaS product are two different disciplines. In a multi-tenant application, a trace is not just a debugging artifact. It is a per-customer record that has to be filterable, attributable, sometimes billable, and occasionally deletable on request. If you cannot answer "show me everything tenant acme-corp ran last Tuesday" in one query, you do not really have observability. You have a firehose.

The good news is that LangSmith was built with exactly this shape of problem in mind. Between metadata, tags, projects, and run naming, you have every primitive you need to slice traces per customer without running a separate LangSmith instance for each one. The bad news is that if you do not decide on a tagging and metadata convention early, you will spend a weekend backfilling and a lot longer regretting the runs you already lost. This article walks through the patterns that hold up in production, the code to implement them, and the mistakes that quietly cost you later.

The Three Isolation Primitives You Actually Have

Before writing any code, it helps to be precise about the tools LangSmith gives you, because people reach for the wrong one constantly. There are three primitives, and they solve different problems.

• Projects are the coarsest grouping. A project is a named bucket of traces, and the LangSmith UI lets you switch between them at the top level. Think of a project as an environment or a major product surface, not as a per-customer container (more on why below).
• Metadata is arbitrary structured key-value data attached to a run. This is where your tenant_id, plan_tier, region, and user_id belong. Metadata is queryable, so you can filter the runs list to metadata.tenant_id = "acme-corp" and get exactly that customer's activity.
• Tags are flat string labels attached to a run. They are cheaper to eyeball than metadata and great for quick visual filtering and faceting, like tenant:acme-corp, env:prod, or feature:contract-summary.

The mental model that keeps teams out of trouble is this: projects for environments, metadata for identity, tags for fast filtering. A run for a paying customer in production might live in the prod-support-agent project, carry metadata {"tenant_id": "acme-corp", "plan": "enterprise"}, and wear tags ["tenant:acme-corp", "env:prod"]. That same trace is now reachable three different ways depending on whether you are debugging, auditing, or building a per-tenant dashboard.

The reason you should resist the urge to create one LangSmith project per customer is that projects do not scale to thousands of tenants gracefully. Project pickers become unusable, cross-tenant analytics get painful, and you lose the ability to ask fleet-wide questions like "what is my p95 latency across all customers." Metadata scales to any number of tenants because it is just a queryable field. Reserve projects for a handful of stable environments and let metadata carry the tenant identity.

Setting Up Tenant-Scoped Tracing With Metadata

Let's make this concrete. The cleanest way to attach tenant identity is at the point where you already know who the request belongs to, which is usually your API handler. LangSmith reads configuration from the run tree, so you pass metadata and tags through the config object on LangChain runnables, or directly through the @traceable decorator if you are using the bare SDK.

Here is the pattern with a LangChain runnable, where the tenant context flows in per request:

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful support assistant for {company}."),
    ("human", "{question}"),
])

model = ChatOpenAI(model="gpt-4o-mini", temperature=0)
chain = prompt | model | StrOutputParser()


def answer_support_question(tenant_id: str, plan: str, region: str,
                            company: str, question: str) -> str:
    # Everything in config.metadata becomes queryable in LangSmith.
    config = {
        "run_name": "support_answer",
        "tags": [f"tenant:{tenant_id}", "env:prod", "feature:support"],
        "metadata": {
            "tenant_id": tenant_id,
            "plan": plan,
            "region": region,
            "app_version": "2026.7.1",
        },
    }
    return chain.invoke(
        {"company": company, "question": question},
        config=config,
    )

The important detail is that metadata and tags set on the top-level config propagate down to every child run in that trace. So the LLM call, the parser, and any nested tools all inherit tenant_id. You tag once at the entry point and the entire tree is attributed. That propagation is what makes this approach practical: you do not have to thread tenant context through every function by hand.

If you are not using LangChain and prefer the raw SDK, the @traceable decorator gives you the same power. You can set static metadata at decoration time and inject dynamic per-request values through the langsmith_extra argument:

from langsmith import traceable
from openai import OpenAI

client = OpenAI()


@traceable(run_type="chain", name="support_answer",
           tags=["env:prod", "feature:support"])
def answer_support_question(tenant_id: str, plan: str, question: str) -> str:
    completion = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a support assistant."},
            {"role": "user", "content": question},
        ],
    )
    return completion.choices[0].message.content


# Per-request tenant identity injected at call time.
answer = answer_support_question(
    tenant_id="acme-corp",
    plan="enterprise",
    question="How do I rotate my API key?",
    langsmith_extra={
        "metadata": {"tenant_id": "acme-corp", "plan": "enterprise"},
        "tags": ["tenant:acme-corp"],
    },
)

The langsmith_extra channel is the escape hatch for values you only know at runtime. Static things that describe the function itself, like feature:support, go in the decorator. Dynamic things that describe the caller, like tenant_id, go in langsmith_extra. Keeping that split clean makes your traces self-documenting.

Propagating Tenant Context Without Threading It Everywhere

Passing tenant_id explicitly into every function is fine for a small surface area, but in a real app your call stack is deep and you do not want tenant identity leaking into every signature. This is where context managers earn their keep. LangSmith provides tracing_context, which lets you set metadata and tags for everything that runs inside a block, so you can establish tenant scope once at the request boundary and let all downstream code inherit it automatically.

from langsmith import tracing_context
from langsmith.run_helpers import get_current_run_tree


def handle_request(tenant_id: str, plan: str, request_body: dict):
    # Establish tenant scope for the whole request lifecycle.
    with tracing_context(
        tags=[f"tenant:{tenant_id}", "env:prod"],
        metadata={"tenant_id": tenant_id, "plan": plan},
    ):
        # Any @traceable function or LangChain call inside here
        # inherits the tenant metadata and tags automatically.
        result = run_agent_pipeline(request_body)
        return result

Everything invoked inside that with block, no matter how deeply nested, gets stamped with the tenant metadata. Your run_agent_pipeline function and its children stay blissfully unaware of multi-tenancy. They just do their job, and the observability layer wraps the whole thing. This is the single biggest ergonomic win when retrofitting tenant isolation onto an existing codebase, because you touch one function, the request handler, instead of hundreds.

In a web framework you would typically wrap this in middleware. For a FastAPI app, resolve the tenant from the auth token in a dependency or middleware layer, then open the tracing_context for the duration of the request. Every LLM operation triggered by that request, whether it is a direct chain call or a background task spawned from it, lands in LangSmith already attributed. The tenant identity becomes ambient rather than an argument you keep forgetting to pass.

One caution: ambient context and background work do not always mix cleanly. If you kick off an async task or hand work to a thread pool, the context may not follow it depending on how the executor is configured. For those cases, capture the tenant metadata into a plain variable before crossing the boundary and re-establish the context inside the worker. Do not assume propagation is magic across every concurrency primitive. Test it.

Filtering and Querying Traces Per Tenant

Attribution is only half the value. The payoff comes when a customer opens a ticket and you need their runs and only their runs, fast. Because you stored tenant_id as metadata, the LangSmith UI lets you filter the runs table with a query. In the filter bar you can write a condition against metadata, and the trace list collapses to a single tenant instantly. That alone justifies the whole setup the first time a support escalation lands.

The same query power is available programmatically, which is how you build per-tenant dashboards, usage reports, and automated audits. The Client.list_runs method accepts a filter expression, and LangSmith's filter DSL exposes helpers for querying metadata and tags:

from langsmith import Client
from datetime import datetime, timedelta

client = Client()

# Pull every top-level run for one tenant in the last 7 days.
tenant_runs = client.list_runs(
    project_name="prod-support-agent",
    filter='and(eq(metadata_key, "tenant_id"), eq(metadata_value, "acme-corp"))',
    start_time=datetime.utcnow() - timedelta(days=7),
    is_root=True,
)

total = 0
total_tokens = 0
for run in tenant_runs:
    total += 1
    if run.total_tokens:
        total_tokens += run.total_tokens

print(f"acme-corp ran {total} traces using {total_tokens} tokens this week")

That loop is the seed of a per-tenant usage report. Sum the token counts and you have the raw material for usage-based billing. Bucket by day and you have a trend line for a customer-facing dashboard. Filter additionally on error status and you have an SLA report showing exactly how reliable that customer's experience has been. Everything downstream flows from the single decision to store tenant_id as queryable metadata at write time.

You can combine conditions too. To find every failed run for a single tenant, you compose the tenant filter with a status filter using the and combinator. To facet across tenants, you drop the tenant condition and group by the metadata field instead. The filter DSL is flexible enough that once your metadata is clean, almost any operational question becomes a query rather than a data-engineering project.

Tags, Run Names, and Making Traces Human-Readable

Metadata is for machines and precise queries. Tags and run names are for the human staring at the trace list at 2 a.m. trying to understand what happened. Both matter, and treating them as an afterthought is a mistake you feel every time you open the UI.

Set a deliberate run_name at each entry point so the trace list reads like a log of intents rather than a wall of identical class names. Compare "RunnableSequence" repeated four hundred times against "support_answer", "contract_summary", and "invoice_extraction". The second list tells you what your system is doing at a glance. Run names cost nothing and pay off on every single debugging session.

Tags are your fast-filter layer. A disciplined tag vocabulary makes the UI's faceting genuinely useful:

def build_run_tags(tenant_id: str, env: str, feature: str,
                   plan: str) -> list[str]:
    # A stable, prefixed tag vocabulary keeps faceting clean.
    return [
        f"tenant:{tenant_id}",
        f"env:{env}",
        f"feature:{feature}",
        f"plan:{plan}",
    ]


tags = build_run_tags(
    tenant_id="acme-corp",
    env="prod",
    feature="contract-summary",
    plan="enterprise",
)

The prefix convention, key:value, is doing real work here. It keeps your tag namespace organized so tenant:acme-corp never collides with feature:acme-something, and it lets you scan a run's tags and immediately parse them. Pick the convention once, write it in your team's docs, and enforce it in a helper function like the one above so nobody hand-types a tag and drifts from the standard. The moment two engineers spell the same concept differently, env:prod versus environment:production, your faceting fractures and you are back to grepping.

A practical rule: put anything you want to filter or facet on in tags, and anything you want to query precisely or read programmatically in metadata. There is overlap, and duplicating tenant_id into both a tag and a metadata field is not just acceptable, it is recommended. The tag gives you one-click filtering in the UI, and the metadata gives you exact programmatic queries. Redundancy across the two costs almost nothing and buys you flexibility.

Isolation, Privacy, and the Data You Should Not Send

Multi-tenancy is not only an organizational concern. It is a data-governance one. The instant you have more than one customer, you have obligations about what leaves each customer's boundary and lands in your observability platform. Traces frequently contain the full prompt and completion, which means they can contain personally identifiable information, secrets, and proprietary customer content. Sending all of that to LangSmith unfiltered is a decision, and it should be a conscious one.

LangSmith supports hiding inputs and outputs so you keep the structural trace, the timing, the token counts, the tenant attribution, without persisting the raw payloads. You can configure this globally through environment variables, or per-client with anonymizer functions that redact fields before anything is transmitted. The pattern is to strip or mask sensitive keys at the boundary:

from langsmith import Client
import re

EMAIL_RE = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")


def redact(data: dict) -> dict:
    # Runs before the payload leaves your process.
    cleaned = {}
    for key, value in data.items():
        if key in {"password", "api_key", "ssn", "credit_card"}:
            cleaned[key] = "[REDACTED]"
        elif isinstance(value, str):
            cleaned[key] = EMAIL_RE.sub("[EMAIL]", value)
        else:
            cleaned[key] = value
    return cleaned


client = Client(
    hide_inputs=redact,
    hide_outputs=redact,
)

Because the anonymizer runs in your process before transmission, the sensitive values never reach LangSmith at all. You still get the tenant metadata, the latency, the token accounting, and the shape of the run for debugging, but the regulated content stays on your side of the wire. For teams with strict compliance requirements this is often the difference between being able to use hosted observability and not.

The deletion story matters too. When a customer invokes their right to be forgotten, your queryable tenant_id metadata is exactly what lets you enumerate and remove their runs. This is another reason to standardize on metadata-based tenancy from day one. A right-to-deletion request against traces you cannot filter by tenant is a nightmare. Against clean metadata it is a scripted list_runs followed by a delete loop. The convention you set for convenience turns out to be the same convention that keeps you compliant.

Common Multi-Tenant Tracing Mistakes and How to Avoid Them

A few failure patterns show up again and again once teams take this into production. Naming them is the fastest way to avoid them.

1. Creating one project per tenant. It feels like clean isolation and it scales terribly. Once you cross a few dozen tenants the project picker is unusable and fleet-wide analytics become impossible. Use metadata for tenant identity and reserve projects for environments.
2. Inconsistent tenant identifiers. If one code path stamps tenant_id as the numeric database ID and another stamps the human-readable slug, your filters silently miss half a customer's runs. Pick one canonical identifier, ideally a stable slug, and use it everywhere. Centralize it in a helper so it cannot drift.
3. Forgetting to propagate context into background jobs. Ambient tracing_context does not always survive a jump into a thread pool or a queued task. Capture the tenant metadata explicitly before the boundary and re-establish it inside the worker, then verify with a real background run that the attribution actually appears.
4. Leaking one tenant's data into another's trace. This happens when tenant context is stored in a long-lived shared object rather than scoped per request. Always resolve tenant identity per request and scope it with a context manager whose lifetime matches the request. Never cache a tenant into module-level state.
5. Tagging everything and querying nothing. Tags without a convention are just noise, and metadata you never query is just storage cost. Decide up front which questions you need to answer, model your metadata and tags to answer exactly those, and prune the rest.
6. Sending raw sensitive payloads by default. The default is to send inputs and outputs verbatim. For a multi-tenant product handling customer data, make redaction a deliberate, reviewed decision rather than something you discover during a security audit.

The through-line across all six is the same: decide your conventions before you have real traffic, encode them in shared helpers so they cannot drift, and treat tenant attribution as a first-class property of every run rather than a debugging nicety you bolt on later. Retrofitting is possible, but it always costs more than getting the convention right the first week.

Bringing It Together Into a Production Setup

Put the pieces side by side and a clean architecture emerges. At the request boundary, you resolve the tenant from the auth context. You open a tracing_context that stamps tenant_id, plan, and env as both metadata and tags for the entire request lifecycle. You set a meaningful run_name at each logical entry point so the trace list reads like intents. You configure your LangSmith client with input and output redaction so regulated content never leaves your process. Downstream, all your chains and agents run untouched, inheriting attribution automatically, while your operational tooling queries list_runs with metadata filters to build per-tenant dashboards, usage reports, and deletion workflows.

None of these primitives is complicated on its own. Metadata is a dictionary. Tags are a list of strings. A context manager is a with block. The skill is in the discipline, choosing a canonical tenant identifier, splitting static from dynamic attribution, deciding what belongs in metadata versus tags, and drawing the privacy boundary deliberately. Get those conventions right and LangSmith gives you genuine per-customer observability on a single shared instance, with the ability to answer both "what did this one customer do" and "how is my whole fleet performing" from the same trace store.

If you want to go deeper than a single article can take you, and see these patterns built end to end against a real multi-tenant app, complete with evaluation datasets scoped per tenant, cost tracking, and production monitoring, that is exactly what the LangSmith Tutorial course on teachyou.ai is built to teach. It walks through the same primitives covered here and then extends them into the full observability and evaluation workflow you need to run LLM features for paying customers with confidence. Start with the conventions in this article, then let the course fill in the operational depth around them.