Function Calling Explained: How LLMs Use Tools

A language model, left to its own devices, is a brilliant conversationalist trapped inside a sealed room. It can reason about the weather, but it cannot look outside. It can describe how to book a flight, but it cannot open the airline's website. It can tell you what your database probably contains, but it has never actually run a query against it. For the first few years of the large language model era, this was simply accepted as the shape of the technology: a model that talks, and a human who acts on what it says. Function calling is the mechanism that broke down that wall. It gives the model a way to say, in a structured and predictable format, "I need to run this specific tool with these specific arguments," and it gives your application a clean way to run that tool and hand the result back. Once you understand this loop, a huge amount of modern AI engineering suddenly clicks into place. Agents, retrieval systems, coding assistants, and workflow automations are all built on top of this one idea. This article walks through what function calling actually is, how the request and response cycle works under the hood, how to write good tool schemas, and where the sharp edges hide.

What Function Calling Actually Means

The phrase "function calling" is slightly misleading, and clearing up the confusion early saves a lot of grief. The model does not call your function. It cannot reach into your process, import your modules, or execute your code. What the model does is produce a structured message that says, in effect, "based on this conversation, the right next step is to invoke the tool named get_weather with the argument city set to Tokyo." Your application code receives that message, decides whether to honor it, runs the real function, and then sends the output back into the conversation so the model can continue.

Think of it as a very disciplined intern who is not allowed to touch any of the equipment. The intern can fill out a precisely formatted request slip: tool name, arguments, done. You, the trusted operator, read the slip, press the buttons, and report back the reading. The intern then interprets that reading and either fills out another slip or writes up the final answer for you. The model supplies intent and interpretation. Your code supplies execution. This division of labor is the entire security and reliability story of function calling, and it is why the pattern is safe enough to put into production even though the model is fundamentally unpredictable.

The reason this works at all is that models are trained to emit these tool requests as well-formed structured data rather than free-flowing prose. You give the model a list of tools it is allowed to request, each described with a name, a human-readable description, and a machine-readable schema for its arguments. The model, having been fine-tuned on exactly this kind of task, learns to pick the right tool and populate its arguments correctly. The output is not "you should probably check the weather in Tokyo"; it is a clean object your parser can consume without guessing.

The Anatomy of a Tool Definition

Everything starts with the tool schema. This is the contract you hand to the model describing what each tool is, what it does, and what shape its inputs take. Most providers converge on a JSON Schema based format because it is precise, widely supported, and easy for both models and validators to work with. A single tool definition typically carries three things: a name the model uses to select it, a description that teaches the model when and why to use it, and a parameters object that defines each argument, its type, and whether it is required.

Here is a representative tool schema for a weather lookup, written in the style most chat completion APIs accept:

{
  "type": "function",
  "function": {
    "name": "get_current_weather",
    "description": "Get the current weather conditions for a specific city. Use this whenever the user asks about temperature, rain, wind, or general conditions for a named location.",
    "parameters": {
      "type": "object",
      "properties": {
        "city": {
          "type": "string",
          "description": "The city name, for example 'Tokyo' or 'San Francisco'"
        },
        "unit": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "The temperature unit to report in. Defaults to celsius if not specified."
        }
      },
      "required": ["city"]
    }
  }
}

Read that schema the way the model reads it. The name is a stable identifier. The description is not decoration; it is the single most important field for getting good behavior, because the model uses it to decide whether this tool is relevant to the current request. The parameters object uses JSON Schema vocabulary: type constrains the shape, properties lists the arguments, enum restricts a value to a fixed set of choices, and required marks which arguments must always be present. When you constrain a field with an enum, you are doing yourself a favor. The model is far more likely to produce a valid value when the valid values are spelled out than when it has to guess an acceptable string.

The provider takes your tool definitions and folds them into the prompt in a structured way before the model ever sees your user's message. You do not write the tools into the prompt text yourself. You pass them as a separate parameter, and the API handles the formatting. This separation matters because it keeps your conversational content clean and lets the provider optimize how tools are presented to the model.

The Request and Response Loop

Function calling is a loop, not a single call, and internalizing the loop is the key to building anything real with it. The sequence has a rhythm that repeats until the model decides it has enough information to answer.

1. You send the conversation plus the list of available tools to the model.
2. The model responds. It either writes a normal text answer, or it emits one or more tool call requests.
3. If it requested a tool, your code executes the corresponding real function with the arguments the model provided.
4. You append the tool's result to the conversation as a new message and send everything back to the model.
5. The model reads the tool result and either answers, or requests another tool. Repeat from step three.

Consider a concrete run. A user asks, "What should I wear in Tokyo today?" The model cannot know the weather, so on the first pass it does not answer. Instead it returns a tool call: invoke get_current_weather with city set to Tokyo. Your application sees this, calls your actual weather service, and gets back something like fifteen degrees celsius and light rain. You package that result and send it back. Now the model has real data. On the second pass it produces a normal text answer: dress warmly, bring a light waterproof jacket, and carry an umbrella. Two round trips, one tool call, and the user gets an answer grounded in real conditions rather than a hallucinated guess.

Here is what a tool call looks like when it comes back from the model, and what you send back after running the tool. First, the model's response containing the tool call:

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_a1b2c3",
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "arguments": "{\"city\": \"Tokyo\", \"unit\": \"celsius\"}"
      }
    }
  ]
}

Notice that the arguments value is a JSON string, not a JSON object. This trips people up constantly. You have to parse that string before you can use it. Notice also the id field. That identifier is how the model tracks which result belongs to which request, which matters enormously when the model fires off several tool calls at once. After you run the function, you reply with a tool result message that references the same id:

{
  "role": "tool",
  "tool_call_id": "call_a1b2c3",
  "content": "{\"temperature\": 15, \"unit\": \"celsius\", \"conditions\": \"light rain\"}"
}

The tool_call_id stitches the result back to the request that asked for it. You append this message to the conversation history and send the whole thing back. The model now has everything it needs to write its final reply. This append-and-resend pattern is the beating heart of every agent framework you will ever use, no matter how much abstraction is layered on top.

Parallel And Sequential Tool Use

Early tool-calling implementations could request only one function at a time. Modern models are smarter about it and will often request several tools in a single turn when those calls are independent of one another. If a user asks for the weather in three different cities, a capable model returns three tool calls in one response, each with its own id. Your code can run all three in parallel, collect the results, and send them back together. This is a real latency win. Instead of three sequential round trips to the model, you make one, fan out the work on your side, and return.

Sequential tool use is the other pattern, and it appears when one tool's output feeds the next tool's input. Suppose a user asks, "How many open pull requests does the busiest contributor to our main repo have?" The model might first call a tool to find the busiest contributor, wait for that name to come back, and only then call a second tool to count that person's open pull requests. It cannot parallelize these because the second call literally depends on the answer to the first. The model handles this dependency reasoning for you, deciding turn by turn what it needs next. Your job is simply to keep running the loop, executing whatever the model asks for and feeding results back, until the model stops asking and starts answering.

The practical implication is that your tool-execution code should be built to handle a list of tool calls per turn, not a single one. Iterate over every tool call in the response, run each, and collect all the results before you resend. Assuming there will only ever be one call is a bug waiting to surface the first time the model decides to batch its requests.

Writing Schemas The Model Understands

The quality of your tool schemas determines the quality of your tool use, full stop. A model can only be as good at picking and populating tools as your descriptions allow. The most common failure mode in production is not the model calling a tool incorrectly; it is the model failing to call a tool when it should, or calling the wrong one, because the schema did not make the right choice obvious.

Treat the description field as prompt engineering, because that is exactly what it is. A weak description says "gets user data." A strong description says "Retrieves the profile record for a single user by their numeric account ID. Returns name, email, signup date, and subscription tier. Use this when you need details about a specific known user, not for searching or listing users." The stronger version tells the model precisely when the tool applies and, just as importantly, when it does not. Boundaries prevent misuse. If two of your tools do similar things, spell out the difference in both descriptions so the model can tell them apart.

A few habits consistently pay off when writing schemas:

• Name tools with clear verbs and nouns, like search_orders or cancel_subscription, so the name alone hints at the behavior.
• Describe every parameter, not just the tool. A parameter description like "The ISO 8601 date to filter from, inclusive" removes ambiguity that would otherwise produce malformed input.
• Use enum for any argument with a fixed set of valid values. This turns a guessing problem into a selection problem.
• Mark only the truly mandatory arguments as required. Overloading required forces the model to invent values it does not have.
• Keep the argument surface small. A tool with three well-chosen parameters is easier for the model to use correctly than one with twelve.

Here is a richer schema that shows these principles working together, this time for a tool that searches support tickets:

{
  "type": "function",
  "function": {
    "name": "search_support_tickets",
    "description": "Search the support ticket system for tickets matching a query. Use this to find existing tickets before creating a new one, or to look up the status of a customer issue. Returns up to 20 matching tickets ordered by most recent.",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "Free-text search terms, such as a keyword from the ticket subject or body"
        },
        "status": {
          "type": "string",
          "enum": ["open", "pending", "resolved", "closed"],
          "description": "Optional filter to only return tickets with this status"
        },
        "priority": {
          "type": "string",
          "enum": ["low", "medium", "high", "urgent"],
          "description": "Optional filter to only return tickets at this priority level"
        }
      },
      "required": ["query"]
    }
  }
}

Every field here does a job. The description tells the model both what the tool returns and the two situations it is meant for. The optional filters are constrained with enums so the model never invents a status like "in progress" that your system does not recognize. Only query is required, which reflects reality: you can search with just a keyword, but the filters are genuinely optional refinements.

Handling The Results Your Code Sends Back

The result you hand back to the model is as much a part of the design as the schema you send in. Models reason over whatever text you put in the tool result message, so the shape and content of that text directly shape the final answer. Return structured, minimal, relevant data. If your weather API returns fifty fields of atmospheric telemetry, do not dump all fifty back into the conversation. Extract the handful the model actually needs and return those. Every extra token you send back costs money, consumes context, and gives the model more opportunity to fixate on something irrelevant.

Errors deserve special care. When a tool fails, do not silently swallow the failure or crash your loop. Send the model a clear, honest error message as the tool result. If a user asks to cancel an order that does not exist, return something like a message stating the order id was not found and suggesting the user verify the number. The model will read that and respond gracefully, asking the user to double-check the order id rather than pretending the cancellation succeeded. Models are surprisingly good at recovering from well-described failures, and surprisingly bad at recovering from failures you hide from them.

Keep the format of your results consistent across calls. If you return JSON one time and a plain sentence the next, you make the model's job harder for no benefit. Pick a convention, usually compact JSON, and stick to it. Consistency lets the model build a stable mental model of what your tools return, which improves reliability over long conversations with many tool calls.

Security, Trust, And The Human In The Loop

The single most important thing to remember about function calling is that the model's request to call a tool is exactly that: a request. It is not a command your code is obligated to obey. This distinction is the foundation of safe tool use, and forgetting it is how people build systems that do dangerous things.

Because the model can be steered by the content of the conversation, including content that arrives from untrusted sources like web pages, emails, or user uploads, you must treat every tool call as potentially adversarial. A malicious instruction buried in a document the model reads could, in principle, cause the model to request a destructive tool. Your defense is that your code sits between the request and the execution. You decide what is allowed.

• Validate every argument before executing. Check types, ranges, and permissions against your own rules, never trusting the model's output blindly.
• Scope tool permissions tightly. A tool that can read data is far less dangerous than one that can delete it. Grant the minimum a task requires.
• Require human confirmation for high-stakes actions. Sending money, deleting records, or emailing customers should pause for an explicit human approval rather than firing automatically.
• Log every tool call and its result. When something goes wrong, a complete trace of what the model requested and what your code did is invaluable.

The mental model to hold onto is that the language model is an untrusted planner and your code is the trusted executor. The planner is creative and capable but fundamentally unpredictable. The executor is boring, deterministic, and where all your safety guarantees actually live. Good function-calling systems put the interesting judgment in the model and the enforcement in code.

From Function Calling To Agents

Once the request-and-response loop is comfortable, agents stop looking like magic and start looking like a natural extension of what you already understand. An agent is, at its core, this same loop running until a goal is met rather than stopping after a single exchange. You give the model a set of tools and an objective, and you let it call tools, read results, and call more tools in a cycle that continues until it declares the task complete.

Everything you have learned scales directly. The tool schemas are the same. The append-and-resend mechanics are the same. The security discipline is the same and, if anything, matters more because an autonomous loop can take many actions without a human reviewing each one. What changes is the control logic around the loop. You add a stopping condition so the agent does not run forever. You add a step limit as a safety net. You may add memory so the agent remembers what it has already tried. But peel back the orchestration and you find the exact function-calling cycle described in this article, turning over and over.

This is why function calling is worth learning properly rather than treating it as a black box behind some framework. Frameworks come and go, and each one wraps the loop in its own vocabulary of chains, graphs, and executors. Underneath every one of them is a model emitting structured tool requests, code running real functions, and results flowing back into the conversation. Understand that primitive deeply and you can read any agent framework's source, debug any tool-use failure, and design systems that behave predictably even when the model does not.

If you want to go from understanding this loop to building production systems on top of it, structured practice makes the difference. The AI Engineering Roadmap course on teachyou.ai walks through function calling, tool design, agent loops, retrieval, evaluation, and deployment in a hands-on sequence, taking you from writing your first tool schema to shipping a reliable multi-tool agent. The concepts in this article are the foundation. The course is where you turn them into something real.