LangSmith Trace Filtering: Finding the Runs That Matter

Your LLM app has been in production for two weeks and your LangSmith project now holds tens of thousands of traces. Somewhere in that pile is the one conversation where your agent looped four times, burned a fortune in tokens, and returned an empty answer to a paying customer. Scrolling will not find it. Sorting by date will not find it. What finds it is LangSmith trace filtering: a small query language and a set of UI and SDK tools that let you slice a mountain of runs down to the handful that actually explain your problem. This guide covers how filtering works in the UI, how the filter query language is structured, how to filter by tags, metadata, feedback, latency, and errors, and how to do all of it programmatically so your debugging workflow scales with your traffic.

Why Trace Filtering Becomes Urgent Faster Than You Expect

When you first wire up LangSmith, the traces view feels manageable. You run a prompt, you click the trace, you inspect the chain. Every run is one you personally triggered, so finding it is trivial. Observability at that stage is just "look at the most recent row."

Production changes the math completely. A modest app serving a few hundred users can generate thousands of root runs a day, and each root run fans out into child runs: retriever calls, tool invocations, LLM calls, parser steps. A single agent conversation might produce thirty or forty runs in one trace tree. Within days, the project view becomes a firehose, and the questions you need to answer are never about the most recent row. They are questions like:

• Which runs failed with an error in the last six hours?
• Which traces took longer than ten seconds end to end?
• What did user 4821 actually see when they complained about a wrong answer?
• Which runs got a thumbs-down from users this week?
• Which calls used the fallback model instead of the primary one?

Every one of these is a filtering question. Teams that never learn LangSmith trace filtering end up doing one of two bad things: they eyeball a random sample of traces and hope it is representative, or they stop looking at traces entirely and debug from application logs, throwing away the structured tree view that made them adopt LangSmith in the first place. Learning the filter system is the difference between observability as decoration and observability as a daily debugging tool.

Traces, Runs, and What You Are Actually Filtering

Before writing filters, it helps to be precise about the objects involved, because LangSmith filters operate on runs, and the word "trace" gets used loosely.

A run is a single unit of work: one LLM call, one tool execution, one chain invocation, one retriever query. Every run has a type such as llm, chain, tool, or retriever, plus a name, inputs, outputs, timing information, token counts where applicable, an error field, tags, and metadata.

A trace is the full tree of runs produced by one top-level invocation. The run at the top of that tree is the root run. When you call an agent once, you get one trace, one root run, and potentially dozens of nested child runs beneath it.

This distinction matters because the runs table in a LangSmith project shows root runs by default, but filters can target any level. If you filter for run_type equals llm, you are searching child runs deep inside trace trees. If you filter on end-to-end latency, you usually care about the root run, because that is what represents the user's actual wait time. A lot of early confusion with LangSmith trace filtering comes from writing a correct filter at the wrong level of the tree: for example, filtering root runs for run_type equal to llm in an agent project and getting nothing back, because the roots are all chains and the LLM calls live underneath them. Keep the tree model in your head and the filter behavior becomes predictable.

Filtering in the UI: The Fast Path

The tracing project view in LangSmith has a filter bar that covers most day-to-day needs without writing any query syntax by hand. You add conditions from a dropdown of attributes, and the UI composes them into a query for you. The attributes you will reach for constantly:

• Status to show only errored runs or only successful ones.
• Name to isolate a specific chain, node, or tool by the name it was traced under.
• Latency with greater-than or less-than comparisons to surface slow runs.
• Tokens to find expensive calls by total, prompt, or completion token counts.
• Time range to bound the search to a deploy window or an incident window.
• Tags and metadata to slice by environment, user, model, or any dimension you attached at trace time.
• Feedback to find runs scored by users or evaluators.

Two UI features deserve special attention because people miss them. The first is full-text search, which scans run inputs and outputs for a string. When a user pastes a screenshot of a bad answer into your support channel, copying a distinctive phrase from that answer into full-text search is usually the fastest possible route to the exact trace. The second is the ability to click into a trace and then filter within the trace tree itself, which matters when a single agent conversation has forty runs and you only care about the tool calls.

Multiple conditions in the filter bar combine with AND semantics, which is what you want most of the time: errored runs AND in the last hour AND named generate_answer. When you need OR logic or negation, you drop into the raw query language, which the UI also accepts directly in an advanced filter box.

The Filter Query Language: Small but Sharp

Under every UI filter is LangSmith's filter query language, a compact functional syntax that you can write by hand in the UI or pass as a string through the SDK and API. It is built from comparison functions wrapped around attribute names and values, composed with logical operators. The core comparators:

• eq(attribute, value) and neq(attribute, value) for equality and inequality.
• gt, gte, lt, lte for numeric and time comparisons.
• has(attribute, value) for membership checks against list-like fields such as tags.
• search(text) for full-text search over run content.
• and(...), or(...), not(...) for composition, each taking other expressions as arguments.

A few concrete expressions show the shape of the language. To find failed runs, filter on error status:

• eq(status, "error") returns runs that raised an exception.
• gt(latency, "5s") returns runs slower than five seconds.
• gt(total_tokens, 4000) returns token-hungry calls.
• has(tags, "production") returns runs tagged with your production label.
• search("refund policy") returns runs whose inputs or outputs mention that phrase.

Composition is where it gets genuinely useful. Suppose you want slow or expensive runs, but only failed ones from your RAG chain:

and(
  eq(name, "rag_pipeline"),
  eq(status, "error"),
  or(
    gt(latency, "10s"),
    gt(total_tokens, 6000)
  )
)

That single expression answers a question that would take an unpleasant amount of clicking otherwise: show me every failure of this specific chain that was also slow or costly, because those are the ones most likely to be timeout or context-overflow related rather than ordinary bugs.

Time-based filtering uses ISO 8601 timestamps against start_time, so gt(start_time, "2026-07-01T00:00:00Z") bounds a search to everything after a deploy. Combined with a status condition, this is the canonical "did the release break anything" query, and it takes about fifteen seconds to write once the syntax is familiar.

Tags and Metadata: Filtering Only Works If You Planned for It

Here is the uncomfortable truth about LangSmith trace filtering: the built-in attributes get you a long way, but the filters that answer business questions depend entirely on what you attached to your runs when you created them. LangSmith cannot filter by customer tier, feature flag, prompt version, or A/B test arm unless you put that information on the trace. Filtering is a write-time discipline as much as a read-time skill.

Tags are simple string labels, best used for low-cardinality dimensions: environment names, release channels, coarse feature areas. Metadata is a key-value dictionary, appropriate for higher-cardinality values like user identifiers, session identifiers, model names, and prompt versions. With the LangSmith Python SDK you attach both at trace time:

• With the @traceable decorator, pass tags=["production", "checkout"] and metadata through the runtime config.
• With LangChain runnables, pass config={"tags": ["production"], "metadata": {"user_id": "4821", "prompt_version": "v3"}} on invocation.

On the query side, tag filtering uses has(tags, "checkout"), while metadata filtering matches key and value together, in the form and(eq(metadata_key, "user_id"), eq(metadata_value, "4821")). The UI exposes the same thing through its metadata filter option, and once a metadata key has appeared in your project the filter dropdown will offer it for autocomplete.

A pragmatic convention worth adopting on day one: always attach an environment tag, a session or conversation identifier, a stable user identifier, and a prompt or config version to every root run. That small set of dimensions covers the overwhelming majority of production investigations. When someone asks "what happened in that conversation," you filter by session id and read the whole story in order. When someone asks "did the new prompt help," you filter by prompt version and compare feedback scores. None of that is possible retroactively, so instrument before you need it.

Programmatic Filtering with the SDK

The UI is where you explore; the SDK is where you operationalize. The Client.list_runs method accepts the same filter query language as a string, alongside convenience parameters for common cases. This is how you build cost reports, feed failure analyses, export runs into datasets, or wire LangSmith data into your own dashboards.

from datetime import datetime, timedelta, timezone
from langsmith import Client

client = Client()
since = datetime.now(timezone.utc) - timedelta(days=1)

# Failed root runs in the last 24 hours
failed = client.list_runs(
    project_name="support-agent-prod",
    is_root=True,
    error=True,
    start_time=since,
)

for run in failed:
    print(run.id, run.name, run.error)

# Slow, token-heavy LLM calls tagged production, using the filter language
expensive = client.list_runs(
    project_name="support-agent-prod",
    run_type="llm",
    filter='and(gt(latency, "8s"), gt(total_tokens, 4000), has(tags, "production"))',
    start_time=since,
)

# Traces for one user, matched on metadata
user_runs = client.list_runs(
    project_name="support-agent-prod",
    is_root=True,
    filter='and(eq(metadata_key, "user_id"), eq(metadata_value, "4821"))',
    start_time=since,
)

Notice the split between the keyword conveniences (is_root, error, run_type, start_time) and the filter string. You can express nearly everything in the filter string alone, but the keyword arguments make common queries readable, and the two combine with AND semantics. The method returns an iterator that pages through results for you, so the same code works whether the query matches ten runs or ten thousand; just be deliberate about time bounds on large projects, because an unbounded scan over months of traces is slow for you and wasteful for your rate limits.

The SDK route also unlocks workflows the UI cannot do at all. A short script that pulls yesterday's thumbs-down runs, extracts their inputs, and appends them to a regression dataset turns your production failures into tomorrow's eval cases. That loop, from filter to dataset to evaluation, is arguably the single highest-leverage habit in LLM engineering.

Trace Filters and Tree Filters: Matching Across the Hierarchy

The subtlest and most powerful part of LangSmith trace filtering is querying across levels of the trace tree. Sometimes the runs you want to see and the condition you want to match live in different places. Two additional parameters handle this: trace_filter, which applies conditions to the root run of the trace, and tree_filter, which applies conditions to any run anywhere in the trace tree, while filter continues to constrain the runs actually returned.

Concrete scenarios make the distinction click:

1. You want the individual LLM calls, but only from conversations that received negative user feedback. Feedback is attached to the root run, so you set filter='eq(run_type, "llm")' and trace_filter='and(eq(feedback_key, "user_score"), eq(feedback_score, 0))'. The result is exactly the model calls inside the bad conversations, which is where the answer to "why was this bad" usually lives.
2. You want the root runs of every trace where a specific tool was invoked somewhere in the tree. The tool call is a child run, so you set filter='eq(is_root, true)' and tree_filter='eq(name, "database_lookup")'. Now you can review whole conversations that exercised the risky tool, not just the isolated tool calls stripped of their context.
3. You want retriever runs, but only from production traffic where the tag was applied at the root. Root condition in trace_filter, run condition in filter.

Without these cross-tree filters, people resort to exporting everything and joining in a notebook, which works but is slow and discourages the quick investigative loop that makes tracing valuable. With them, "show me the prompt sent to the model in every conversation the user hated" is one query. Feedback-based filters deserve a special mention here: eq(feedback_key, "correctness") combined with a score comparison lets you navigate directly from evaluation results to the underlying traces, closing the gap between measuring quality and understanding it.

Filtering Recipes for Real Debugging Sessions

Patterns recur. Here is a working set of recipes that covers most production investigations, expressed in the filter language so they work in both the UI advanced box and the SDK.

• Post-deploy triage. and(eq(status, "error"), gt(start_time, "2026-07-03T14:00:00Z")) with the timestamp set to your deploy time. Run it ten minutes after every release. Compare the error names against the previous window to spot regressions instantly.
• Latency outliers. and(eq(is_root, true), gt(latency, "15s")). Open a few matches and look at the trace waterfall: the slow span is usually a retry storm, a cold retriever, or an oversized prompt, and the tree view makes each signature visually distinct.
• Token budget violations. gt(total_tokens, 8000) on run_type llm. If matches cluster on one chain, your context assembly is unbounded somewhere, often a chat history that never truncates.
• Silent failures. and(eq(status, "success"), search("I don't know")) or whatever apology phrase your app produces. These runs raised no error but delivered no value, and they never show up in error-based monitoring. This is the single most underused filter category.
• One user's whole story. and(eq(metadata_key, "session_id"), eq(metadata_value, "abc-123")) sorted by start time gives you the full conversation timeline for a support escalation.
• Fallback model audit. If you record the served model in metadata, filtering on it reveals how often your fallback path actually fires, which is otherwise invisible until the invoice arrives.

Save the ones you use repeatedly. LangSmith lets you save filters as named views on a project, so "prod errors last 24h" and "slow roots" become one click for the whole team. Saved views also standardize vocabulary across teammates, which sounds trivial but ends the situation where two engineers investigate "slow runs" with two different thresholds and reach two different conclusions.

From Filters to Automations and Monitoring

A filter you run every morning is a filter that should run itself. LangSmith's automation rules take a filter plus a sampling rate and apply an action to every matching run: add to an annotation queue, add to a dataset, or trigger a webhook. This is where trace filtering stops being a debugging tool and becomes infrastructure.

The patterns that pay off quickly:

1. Failure harvesting. A rule matching errored root runs feeds an annotation queue. A teammate spends fifteen minutes a day triaging the queue, labeling root causes. Within weeks you have a taxonomy of real failure modes instead of anecdotes.
2. Regression dataset growth. A rule matching negative-feedback runs appends them to a dataset. Every eval you run afterward is automatically tested against the exact inputs that embarrassed you in production.
3. Alerting on quality, not just uptime. A webhook rule on a filter like high-latency production roots pipes matches into your incident channel, giving you an alert stream defined in LLM-native terms such as tokens, feedback, and tool errors rather than generic HTTP metrics.

The monitoring dashboards in LangSmith apply the same idea in aggregate: charts of error rate, latency percentiles, token usage, and feedback scores can be scoped by tag or metadata, so a well-instrumented project gets per-environment and per-prompt-version dashboards for free. The common thread is that every capability compounds on the same two investments: consistent tagging and metadata at write time, and fluency with the filter language at read time. Skimp on either and the fancy features have nothing to grip.

Keep Practicing: Make Filtering a Reflex

The engineers who get the most out of LangSmith are not the ones who memorized every attribute name. They are the ones for whom filtering is a reflex: a vague complaint arrives, and within a minute they are staring at the exact five traces that explain it. That reflex comes from a handful of habits this article has walked through. Model your data as trees of runs so you filter at the right level. Instrument tags and metadata on day one, before you need them. Learn the small query language, eq, gt, has, search, and, or, well enough to compose it without documentation. Use trace_filter and tree_filter when your question spans levels of the tree. Save the views your team uses daily, and promote the filters you run every morning into automations that run themselves.

None of this requires heroics. It requires a few deliberate hours of practice against a real project, which is exactly what most teams never schedule. If you want a structured path through all of it, from your first traced run to filters, evaluations, datasets, annotation queues, and production automations, our LangSmith Tutorial course on teachyou.ai walks through each capability hands-on, with the same production-shaped scenarios described here. Build the reflex now, while your trace volume is still small enough to forgive mistakes, and future incidents will feel less like archaeology and more like a lookup.