What Is an AI Agent? A Practical Definition for Engineers in 2026

Every few months the term gets stretched until it means nothing — a chatbot with a system prompt gets called an agent, a cron job that pings an LLM gets called an agent, a single function call to GPT gets called an agent. If you're an engineer trying to ship something real, that vagueness is a problem. You need a definition you can code against, not a marketing definition. So let's answer the question precisely: what is an AI agent, in terms you can actually build with.

What is an AI agent, precisely

An AI agent is a system where a language model decides what to do next, in a loop, using tools, until it determines the task is done. That's the whole definition. Strip away the hype and three properties fall out of it:

• The model controls the control flow. In a normal program, you write the if-statements. In an agent, the LLM looks at the current state and decides which branch to take — which tool to call, what to do with the result, whether to continue or stop.
• It runs in a loop, not a single pass. One prompt-in, response-out call is not an agent. An agent observes, acts, observes the result of its action, and decides again. This loop can run for one iteration or fifty.
• It has tools that let it affect or query the world. A model that only generates text is a text generator. A model that can call a search API, run code, write to a database, or hit a webhook — and can decide when to do so — is an agent.

If a system is missing any one of these three, it's something else: a classifier, a RAG pipeline, a chatbot, a workflow. That's not a knock on those systems — most production LLM features today are exactly one of those, and that's often the right call. But calling everything an "agent" makes it impossible to reason about design tradeoffs, so we're going to be strict about the term for the rest of this article.

A useful mental model: an agent is a while-loop around a model, where the model's output is parsed to decide the next side effect. Here's that idea in its most reduced form, in Python:

def run_agent(user_goal, tools, model, max_steps=10):
    messages = [{"role": "user", "content": user_goal}]

    for step in range(max_steps):
        response = model.generate(messages, tools=tools)

        if response.tool_call is None:
            # Model decided it's done — no more actions needed
            return response.content

        # Model chose a tool and produced arguments for it
        tool_name = response.tool_call.name
        tool_args = response.tool_call.arguments
        result = tools[tool_name].execute(**tool_args)

        # Feed the result back so the model can decide the next step
        messages.append({"role": "assistant", "content": response.tool_call})
        messages.append({"role": "tool", "content": result})

    return "Max steps reached without completion"

Notice what's absent from this code: there's no logic anywhere that says "if the user asks X, call tool Y." The model is making that decision every single iteration, based on what it sees in the message history. That delegation of control flow to the model is the entire distinction between an agent and a workflow, and it's worth sitting with because it explains both why agents are powerful and why they're hard to trust.

Agents vs. workflows vs. chatbots

These three get conflated constantly, so let's separate them with a concrete example: an internal tool that helps a support team resolve billing tickets.

A chatbot version of this tool takes a user message, sends it to the model with a system prompt describing tone and scope, and returns a reply. It might do this well — it can be genuinely helpful, encode a lot of domain knowledge, and answer with citations. What it cannot do is look anything up, take an action, or verify its own answer against a real system. Every response is a single forward pass. The bottleneck is that it hallucinates status ("your refund was processed") when it has no way of knowing that.

A workflow (sometimes called a "chain" or "pipeline") fixes the lookup problem, but keeps the control flow in your code. You might write: step 1, classify the ticket type; step 2, if it's a refund request, query the payments table; step 3, if the refund exists, template a response; step 4, if not, escalate. Each step might call an LLM, but you decided the sequence and the branches ahead of time. This is deterministic, testable, and — critically — debuggable, because you can point to exactly which if branch fired. Most "AI features" that ship in production are this pattern, and that's a feature, not a compromise.

An agent version hands the sequencing to the model. You give it tools — look_up_customer, check_refund_status, issue_refund, escalate_to_human — and a goal: "resolve this billing ticket." The model decides which tool to call first, reads the result, and decides what to call next. If the customer describes a scenario you didn't explicitly anticipate — a refund that partially processed because of a currency conversion error — the agent can still reason its way through it by chaining tool calls, because nobody pre-wrote that branch. That's the payoff of agentic control flow: it generalizes to situations your workflow's if statements don't cover.

The tradeoff is exactly as obvious as it sounds. A workflow fails predictably — you know its blind spots because you enumerated its branches. An agent fails unpredictably, because the model can choose a bad sequence of tool calls you never imagined, and it can do so confidently. Use this comparison list as a working checklist:

• Chatbot: single model call, no tools, no state beyond conversation history. Good for Q&A, drafting, explanation.
• Workflow: fixed sequence of steps, possibly with LLM calls inside individual steps, branching logic written by you. Good for well-understood processes with known edge cases.
• Agent: model chooses the sequence of tool calls at runtime, loop continues until the model decides it's done. Good for open-ended tasks where the space of valid approaches is too large to enumerate in advance.

The core loop: perceive, decide, act, observe

Every agent framework you'll encounter — whether it's a hand-rolled loop, LangGraph, a Claude-based tool-use loop, or something built on the OpenAI Assistants pattern — is a variation on the same four-beat cycle:

1. Perceive. The agent reads its current context: the user's goal, the conversation so far, and the results of any previous tool calls.
2. Decide. The model reasons over that context and picks the next action — call a specific tool with specific arguments, or conclude the task is finished.
3. Act. The chosen tool actually executes — a function runs, an API gets called, a query hits a database.
4. Observe. The result of that action is appended back into context, and the loop returns to step 1.

This is sometimes called the ReAct pattern (reason, act) in the research literature, though the exact naming has drifted as the industry has adopted it. What matters for you as an engineer isn't the name, it's that each of these four steps is a place where things can go wrong, and each is a place you can add guardrails. A production agent isn't just "loop until done" — it's that loop plus validation at each seam: validating tool arguments before execution, validating tool outputs before feeding them back, and capping the number of iterations so a confused model doesn't spin forever.

Here's a slightly more realistic version, in TypeScript, showing where those seams live:

type ToolResult = { ok: true; data: unknown } | { ok: false; error: string };

interface Tool {
  name: string;
  description: string;
  parameters: object; // JSON schema
  execute: (args: Record<string, unknown>) => Promise<ToolResult>;
}

async function runAgentLoop(
  goal: string,
  tools: Tool[],
  model: ModelClient,
  maxSteps = 12
): Promise<string> {
  const messages: Message[] = [{ role: "user", content: goal }];

  for (let step = 0; step < maxSteps; step++) {
    const response = await model.generate({ messages, tools });

    if (!response.toolCall) {
      return response.text; // model signaled completion
    }

    const tool = tools.find((t) => t.name === response.toolCall!.name);
    if (!tool) {
      messages.push({ role: "tool", content: `Error: unknown tool ${response.toolCall.name}` });
      continue;
    }

    const result = await tool.execute(response.toolCall.arguments);
    messages.push({ role: "assistant", content: response.toolCall });
    messages.push({
      role: "tool",
      content: result.ok ? JSON.stringify(result.data) : `Error: ${result.error}`,
    });
  }

  throw new Error(`Agent did not finish within ${maxSteps} steps`);
}

The details that separate a toy agent from a production one are almost all in the parts this snippet only gestures at: what happens in tool.execute when the arguments are malformed, what you do when a tool times out, how you decide maxSteps, and what you log at every iteration so you can debug a failure after the fact. None of that is exotic engineering — it's the same discipline you'd apply to any system that calls external services — but it's easy to skip when you're excited about the model picking its own tools.

Memory and state: what the agent actually knows

A question that trips people up early: does the agent "remember" anything between steps, or between sessions? By default, no. The loop above is entirely stateless outside of the messages array — everything the model "knows" at step 7 is whatever got appended to that array in steps 1 through 6. This is worth internalizing because it explains a lot of agent behavior that otherwise looks like magic or like a bug.

There are three distinct kinds of memory worth separating:

• Working memory — the message history within a single run. This is what the loop above manages directly. It's bounded by the model's context window, and long-running agents will eventually need to summarize or truncate older steps to stay within that budget.
• Tool-backed memory — a database, vector store, or file system the agent can read from and write to via tools, which persists across runs. This is how an agent "remembers" a user's preferences across separate conversations: not because the model retained anything, but because a previous run wrote a fact to storage and a read_memory tool fetches it back in.
• Episodic/session memory — a deliberate log of past interactions that gets summarized and re-injected, used for agents that need continuity over days or weeks rather than one sitting.

If you're building something like a personal knowledge assistant that needs to accumulate context over time — notes, decisions, half-finished projects — the memory layer is usually the hardest part of the system, harder than the tool-calling loop itself. That's exactly the territory covered in our Building a Second Brain with AI Agents course, which goes deep on designing memory architectures that don't just dump everything into a giant context window and hope.

A worked example: an agent with two tools

Abstract loops are easy to nod along to and hard to actually picture. Here's a concrete, runnable-in-spirit example: an agent that answers questions about a codebase by searching files and reading them, deciding for itself which files are relevant.

from dataclasses import dataclass

@dataclass
class SearchResult:
    path: str
    snippet: str

def search_files(query: str) -> list[SearchResult]:
    # Real implementation: grep, embeddings search, or an index
    ...

def read_file(path: str) -> str:
    with open(path) as f:
        return f.read()

tools = {
    "search_files": {
        "fn": search_files,
        "schema": {
            "name": "search_files",
            "description": "Search the codebase for files matching a query",
            "parameters": {"query": {"type": "string"}},
        },
    },
    "read_file": {
        "fn": read_file,
        "schema": {
            "name": "read_file",
            "description": "Read the full contents of a file at a given path",
            "parameters": {"path": {"type": "string"}},
        },
    },
}

question = "Where do we validate webhook signatures, and is it timing-safe?"
answer = run_agent(question, tools, model=my_model, max_steps=8)

Trace what actually happens when this runs. Step 1: the model sees the question and, having no file contents yet, calls search_files with something like "webhook signature validation". Step 2: it gets back three candidate paths and reads the most likely one with read_file. Step 3: it sees the file imports a naive == comparison instead of a constant-time comparison, and either calls search_files again to check for a shared crypto utility, or concludes it has enough information. Step 4: it returns an answer citing the specific file and line, and flags the timing-attack risk.

Nowhere did you write "if the question mentions webhooks, read the webhook handler file." The model inferred the search query, chose which result to open, and decided when it had enough evidence — that's the agentic property in action, on a task simple enough to reason about by hand. Scale this same pattern up and you get coding agents, research agents, and operations agents; the loop doesn't change, only the tool set and the goal do.

Common failure modes

Agents fail in a handful of recognizable ways. Knowing the pattern names makes them much faster to diagnose when you see them in a trace.

• Looping without progress. The model calls the same tool with slightly different arguments repeatedly, without converging — often because a tool's error message doesn't give it enough signal to correct course. Fix: make tool errors specific and actionable ("file not found; did you mean src/auth/webhook.py?" beats a bare False).
• Tool misuse. The model calls a tool with malformed or nonsensical arguments — wrong types, out-of-range values, a path that doesn't exist. Fix: strict schema validation before execution, and return validation errors as tool results rather than crashing the process.
• Premature stopping. The model decides it's "done" after a shallow pass, especially under prompts that reward brevity. Fix: an explicit checklist in the system prompt, or a verification step (sometimes another model call) that checks the goal was actually satisfied before returning.
• Runaway loops and cost blowouts. Without a hard step limit and a token/cost budget, a confused agent can burn enormous amounts of API spend chasing a task it can't complete. Fix: max_steps, wall-clock timeouts, and cost ceilings enforced in code, not requested politely in the prompt.
• Context poisoning. A bad tool result — a scraped page full of junk, an error stack trace — gets appended to the message history and derails every subsequent decision, because the model keeps attending to it. Fix: truncate and sanitize tool outputs before they go back into context.
• Overconfident irreversible actions. The agent calls a tool that sends an email, charges a card, or deletes a record, based on a misread of the goal. Fix: gate irreversible or high-stakes tools behind human confirmation, or make them dry-run by default.
• Silent scope creep. Given a broad goal and a rich tool set, the agent does more than asked — modifying files it wasn't asked to touch, querying data outside the intended scope. Fix: narrow tool permissions per task rather than handing every agent the full toolbox.

If you take one thing from this list: almost every fix is about constraining the environment around the model, not about making the model "smarter." Prompting can reduce these failure rates, but it cannot eliminate them, because the model is making a probabilistic choice at every step. Durable agent systems assume failures will happen and build recovery paths — retries, validation, human checkpoints — rather than betting everything on the model always choosing correctly.

When NOT to build an agent

This is the section most articles on this topic skip, because "just build an agent" is a better story than "you probably don't need one." But the honest engineering answer is that agents are the right tool for a specific, narrower slice of problems than the current hype implies.

Don't reach for an agent when:

• The task has a known, finite set of steps. If you can write down the branches — "if category is X do A, if category is Y do B" — a workflow will be more reliable, cheaper to run, and far easier to debug than an agent re-deriving that same branch logic on every call.
• Latency matters and the task is simple. Every additional loop iteration is another model call. A single well-crafted prompt that extracts a field or classifies a message will always be faster than an agent that "decides" to do the same thing after two tool calls.
• Mistakes are expensive and hard to reverse. If a wrong action means money moves, data gets deleted, or a customer gets a wrong legal answer, the nondeterminism of agentic control flow is a liability, not a feature. Use a workflow with an LLM doing narrow, checked sub-tasks, and put a human in the loop before anything irreversible.
• You need reproducible behavior for compliance or testing. Agents make different tool-call sequences on different runs, even on ostensibly identical inputs, because model sampling isn't fully deterministic and small context differences compound. If your test suite needs to assert "given input X, exactly these three things happen," a workflow gives you that; an agent usually doesn't without heavy constraints.
• The "tools" you'd give it are really just one API call. If the whole task is "take this input, call this one endpoint, format the output," you don't need a model deciding whether to call it — you need a function. Don't dress up a single API call as an agent because the term is fashionable.
• You haven't defined what "done" looks like. An agent loop needs a stopping condition the model can recognize. If you can't articulate what success looks like well enough to put it in a prompt, the agent won't recognize it either, and you'll get either premature stops or runaway loops.

A good rule of thumb: start with the simplest thing that could work — a single prompt, then a workflow — and only reach for full agentic control flow when you can point to a real requirement that a fixed sequence of steps can't satisfy. Most teams that jump straight to "let's build an agent" end up rebuilding a workflow anyway, just with more latency and less predictability, because they never actually needed the model to control the branching.

Where agents genuinely earn their complexity

To be fair to the pattern: there are real problem classes where nothing else works as well. Open-ended research tasks, where the number of useful next steps depends entirely on what the last search turned up. Coding tasks, where "read this file, then decide whether to read another or make an edit" cannot be fully enumerated ahead of time. Long-horizon operational tasks — provisioning infrastructure, triaging an incident, planning a multi-leg itinerary — where the right sequence of actions genuinely depends on intermediate results in a way a flowchart can't capture without becoming an unmanageable tangle of branches.

The signal to look for is this: if you tried to draw the workflow as a flowchart and the number of boxes and arrows would be enormous, or you keep discovering new edge-case branches every week, that's a task where letting the model control the sequencing starts to pay for itself. If the flowchart fits comfortably on one page, it probably shouldn't be an agent.

Getting hands-on

Reading about the loop is a reasonable start, but the failure modes above — tool misuse, runaway loops, context poisoning — don't really click until you've watched them happen in a trace you built yourself. The fastest way to build the right intuition is to wire up a minimal agent with two or three tools, deliberately break something (feed it a bad tool result, remove the step limit, give it an ambiguous goal), and watch what it does.

If you want a structured path through that instead of debugging it cold, our Introduction to AI Agents course is free and starts from exactly this definition — the loop, the tools, the failure modes — before building up to real projects with Pramod Dutta and Ira Menon. It's the natural next step after this article: less "what is an agent" and more "here's one running, break it with me."