How Agents Choose Tools: Selection and Routing

Agent tool selection is the process by which a large language model decides which function to call, with what arguments, given a user request and a list of available tools. Get it wrong and an agent calls the wrong API, hallucinates a parameter, or burns its context window loading tool schemas it never needed. Get it right and the same agent reliably reaches for a search tool, a database query, or a file write at exactly the moment the task calls for it. This article walks through how tool selection actually works under the hood, why it degrades as tool catalogs grow, and the routing patterns (static lists, semantic search, deferred loading, subagent delegation) that keep it working past a handful of tools.

What tool selection actually is

Modern LLM APIs expose "tool use" or "function calling" as a first-class feature. You send the model a list of tool definitions, each with a name, a description, and a JSON Schema for its parameters, alongside the conversation. The model, at inference time, either replies in plain text or emits a structured tool call: a tool name plus a JSON object of arguments. That's the entire mechanism. There is no separate classifier, no routing model bolted on the side by default. Selection is just the base model doing next-token prediction over a vocabulary that happens to include "call this tool with these arguments."

That matters because it means tool selection quality is bounded by the same things that bound any LLM output: how well the prompt (here, the tool description) disambiguates the choice, how much competing context is in the window, and how much the schema itself steers the model toward valid arguments. A vague tool description is exactly as damaging as a vague instruction in a system prompt. A tool named process with a description "handles the request" gives the model nothing to route on. A tool named refund_order with a description "Issue a refund for a completed order. Use only after the customer confirms the order ID and reason. Does not cancel active subscriptions" gives the model a decision boundary.

Why more tools make selection worse

The naive assumption is that giving an agent more tools makes it more capable. Up to a point, yes. Past that point, accuracy drops, not because the model gets dumber, but because of three compounding effects:

1. Context dilution. Every tool definition (name, description, full JSON Schema with nested properties) sits in the context window on every single turn, whether or not it's relevant. A catalog of 80 tools with verbose schemas can burn 15,000-30,000 tokens before the user has said a word. That's context the model isn't using to reason about the actual task.
2. Semantic overlap. Once you have more than a couple dozen tools, some will look similar to the model. search_docs and search_knowledge_base and query_help_center might be three genuinely different tools with different backends, but from the model's perspective they're near-duplicate token sequences competing for the same intent. This is where wrong-tool calls come from most often, not from the model failing to understand the user's request, but from failing to disambiguate between tools that describe themselves similarly.
3. Latency and cost. Larger tool lists mean larger prompts, which means slower time-to-first-token and higher token cost per call, on every turn, even when nine of ten tools go unused that session.

Anthropic and OpenAI's own tool-use guidance converge on the same number: once an agent's tool count climbs into the dozens, flat static lists stop being reliable and you need a routing layer in front of the raw catalog.

Pattern 1: static allowlist (the default, and when it's fine)

For agents with a small, stable toolset (roughly under 15-20 tools), just send the whole list every turn. This is the default behavior of most agent frameworks and it's the right choice until you have evidence it isn't. Two things earn their keep here:

• Tight descriptions. Write tool descriptions the way you'd write a docstring for a junior engineer who will never see your code, only the doc: what it does, when to use it, when *not* to use it, and what it returns.
• Strict schemas. Use JSON Schema enum, required, and additionalProperties: false aggressively. The tighter the schema, the less room the model has to guess wrong.

Example tool definition (Anthropic Messages API shape):

{
  "name": "get_order_status",
  "description": "Look up the current status of a single order by its order ID. Returns status, tracking number if shipped, and estimated delivery date. Does not modify the order. Use search_orders first if you only have a customer name or email.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "The order ID, formatted like ORD-12345."
      }
    },
    "required": ["order_id"],
    "additionalProperties": false
  }
}

Notice the description does routing work inline: it tells the model when to reach for a *different* tool (search_orders) instead. That single sentence prevents a class of wrong-tool calls without any extra infrastructure.

Pattern 2: deferred (lazy) tool loading

Once the catalog grows past what fits comfortably in context, the next step isn't a smarter model, it's not sending the full catalog at all. Deferred loading keeps a lightweight index of tool names and one-line summaries in context, and exposes a single meta-tool, commonly something like a tool_search or find_tools call, that the model uses to fetch the full schema for the two or three tools it actually needs for the current step.

The flow looks like this:

Turn 1 (system context): 200 tools listed by name + 5-word summary only (~2k tokens)
Model:  "I need to send a Slack message" -> calls tool_search("send slack message")
System: returns full schema for slack_post_message (and maybe 2 near matches)
Model:  now calls slack_post_message with real arguments

This is the same idea behind retrieval-augmented generation, applied to tools instead of documents: don't put everything in context, put an index in context and let the model pull what it needs. The tradeoff is an extra round trip per unfamiliar tool, so it costs a little latency on the first use of any given tool, but it keeps steady-state context small and scales to hundreds of tools without degrading selection accuracy on the ones actually in play.

A minimal Python sketch of the router side of this pattern:

import json

TOOL_INDEX = {
    "send_slack_message": "Post a message to a Slack channel or user.",
    "create_jira_ticket": "File a new Jira issue in a given project.",
    "query_database": "Run a read-only SQL query against the analytics warehouse.",
    # ... hundreds more, name + one-liner only
}

FULL_SCHEMAS = load_full_schemas()  # loaded from disk/DB, not sent to the model up front

def tool_search(query: str, max_results: int = 5):
    # naive keyword match; swap for embedding search at scale
    query_terms = query.lower().split()
    scored = []
    for name, summary in TOOL_INDEX.items():
        text = f"{name} {summary}".lower()
        score = sum(1 for term in query_terms if term in text)
        if score:
            scored.append((score, name))
    scored.sort(reverse=True)
    top = [name for _, name in scored[:max_results]]
    return {name: FULL_SCHEMAS[name] for name in top}

The model calls tool_search, gets back full JSON Schemas for a short list of candidates, and on its next turn calls the real tool by name with real arguments. Only tool_search itself needs to be defined in the base tool list; everything else is fetched on demand.

Pattern 3: semantic routing over a tool catalog

Keyword matching (as in the sketch above) works for small or well-labeled catalogs. Past a few hundred tools, or when tool names don't share vocabulary with how users phrase requests, embed each tool's name and description once, store the vectors, and route by cosine similarity against the embedded user request. This is the same infrastructure as a document retrieval pipeline, just pointed at tool descriptions instead of paragraphs.

from typing import List, Tuple

def embed(text: str) -> List[float]:
    # call your embedding model here
    ...

def semantic_route(user_query: str, tool_vectors: dict, top_k: int = 5) -> List[str]:
    query_vec = embed(user_query)
    scored: List[Tuple[float, str]] = []
    for tool_name, vec in tool_vectors.items():
        score = cosine_similarity(query_vec, vec)
        scored.append((score, tool_name))
    scored.sort(reverse=True)
    return [name for _, name in scored[:top_k]]

Precompute tool_vectors once when the catalog changes, not per request. This is the pattern behind most "tool marketplaces" inside agent platforms today: hundreds or thousands of registered tools, a handful surfaced per turn based on semantic proximity to the task at hand.

Pattern 4: routing to subagents instead of tools

Sometimes the unit of routing shouldn't be an individual function, it should be a whole specialized agent with its own tool loadout. This is the orchestrator/subagent pattern: a top-level agent holds a small number of "dispatch" tools, each of which hands a task off to a subagent that has deep access to a narrow toolset (a coding subagent with file and shell tools, a research subagent with search and fetch tools, a data subagent with database tools).

This solves the context-dilution problem structurally rather than through search: the orchestrator never sees the coding subagent's 40 file-manipulation tool schemas, it just sees one delegate_to_coding_agent(task_description) tool. The routing decision at the top level is coarse (which specialist handles this?) and the routing decision inside each subagent is fine-grained but over a small, coherent toolset where semantic overlap is naturally lower because the tools all live in the same domain.

The failure mode to watch for here is over-fragmentation: if every subagent needs to call back into another subagent to finish a task, you've traded tool-selection errors for coordination overhead and added latency without improving accuracy. Subagent boundaries should follow natural task boundaries (frontend vs backend, research vs writing, read vs write access) not be sliced arbitrarily.

Guardrails: what selection alone won't catch

Tool selection is a probabilistic process. Even a well-described, well-scoped tool catalog will occasionally get called wrong, especially under adversarial or ambiguous input. Selection quality is not a substitute for authorization and validation:

• Permission gates on side-effecting tools. Anything that writes, deletes, sends money, or sends a message on a user's behalf should require explicit confirmation or sit behind an allowlist, independent of whether the model "chose" to call it. Treat the model's tool call as a proposal, not an authorization.
• Argument validation before execution. JSON Schema catches malformed shapes but not semantically wrong values (a valid-looking order ID that belongs to someone else). Validate at the tool boundary, not just at the schema boundary.
• Read-only tools first. When two tools could plausibly serve a request and one is read-only, bias the routing (via description language, via ordering, via a lower-friction dispatch path) toward the safer one, and require a second confirming signal before the side-effecting one fires.

Evaluating tool selection

Treat tool selection as a model behavior you test, not one you assume. A minimal eval set:

1. Single-tool precision. For a set of unambiguous prompts, does the agent call the one correct tool with correct arguments? Track this as a straightforward accuracy percentage.
2. Disambiguation under overlap. For prompts designed to sit between two similar tools, does the agent pick the one the description says to prefer? This is where most regressions show up after adding a new tool to an existing catalog.
3. Correct refusal. For prompts that don't map to any available tool, does the agent say so instead of forcing a call to the nearest-sounding one? A model under pressure to "be helpful" will sometimes call a tool that's close-but-wrong rather than say no tool fits.
4. Multi-step routing. For tasks that require calling tool A, reading its result, then choosing between tools B and C based on that result, does the chain hold together, or does the agent lose the thread after the first call?

Run this eval set every time you add, remove, or reword a tool description. Tool descriptions are prompts; they regress like prompts.

FAQ

How many tools can one agent reliably handle in a single turn? There's no universal number because it depends on schema verbosity and how distinct the tools are, but as a working rule: under 15-20 well-differentiated tools in a flat list is comfortable for most current models. Past that, invest in deferred loading or semantic routing rather than trusting the model to sort through a longer flat list.

Is tool selection the same as intent classification? They're related but not identical. Intent classification maps a user utterance to a category. Tool selection maps a model's current reasoning state (which may span several turns and prior tool results) to a specific function call with specific arguments. Tool selection is a strict superset: it includes argument extraction, which classification alone doesn't.

Do I need a separate routing model, or can the same LLM handle routing and execution reasoning? For most agents, the same model handles both, since function calling is a native capability, not a bolted-on layer. A separate lightweight router (a small classifier or embedding search) earns its place only when the catalog is large enough that sending full schemas every turn is itself the bottleneck, as in the deferred-loading pattern above.

What's the most common cause of wrong tool calls in production? Semantically overlapping tool descriptions, not model weakness. Two tools that both plausibly "handle search" or both plausibly "update a record" will get confused regardless of model quality. The fix is almost always rewriting descriptions to state the boundary explicitly ("use X for A, use Y for B, never use X for B because...") rather than swapping models.

Should tool descriptions be written for the model or for the developer reading the code? For the model. It's read on every inference call and directly shapes routing behavior, so it should read like an instruction, not like a code comment. Keep developer-facing explanation (why the tool exists, implementation notes) in code comments or docs, separate from the description field the model actually sees.

How does this connect to context window management generally? Tool schemas are one of several categories of "always-on" context (system prompt, tool definitions, memory files) that compete with the actual conversation for space. The same discipline that applies to trimming system prompts, only include what's needed for this turn, applies to tool catalogs. Deferred loading is just that discipline applied specifically to tools.