Deploying AI Agents on Serverless

Agent serverless deployment sounds simple until you actually try it: you package an agent loop into a function, deploy it, and the first real request times out because the agent needed six tool calls and your platform kills anything running past 30 seconds. This guide walks through the actual mechanics of getting an AI agent running on serverless infrastructure, including the parts that trip people up: timeouts, state, cold starts, and cost control.

We will build a working example on AWS Lambda, then look at how the same pattern maps to Cloudflare Workers and Modal, and cover the operational pieces (logging, secrets, retries) that separate a demo from something you can point real traffic at.

Why Agent Serverless Deployment Is Harder Than Deploying a Function

A typical serverless function does one thing: receives a request, does a bounded amount of work, returns a response. An AI agent is different because it runs a loop. It calls a model, the model decides to call a tool, the tool runs, the result goes back to the model, and this repeats until the model decides it's done. That loop can take anywhere from two seconds to two minutes depending on how many tool calls the task needs.

This creates three problems that don't show up in a normal serverless deployment:

• Variable execution time. A support agent answering a simple question might finish in 3 seconds. The same agent researching a multi-step task might run for 90 seconds across eight tool calls. Your platform's timeout has to accommodate the worst case, not the average case.
• Statefulness across turns. If a user sends a follow-up message, the agent needs its prior conversation and tool results. Serverless functions are stateless by design, so you need an external store for that state.
• Streaming and partial results. Users expect to see tokens as they arrive, not wait 40 seconds for a blank screen. Serverless platforms handle streaming responses differently, and some don't support it well at all.

None of these are blockers. They just mean you cannot treat an agent like a CRUD endpoint and expect it to behave.

Choosing a Serverless Platform for Agent Workloads

Before writing code, pick a platform based on how your agent actually behaves, not on what's trendy.

AWS Lambda is the default choice if you're already on AWS. It supports up to 15 minutes of execution time, response streaming via Lambda Function URLs, and integrates cleanly with Step Functions if you need to orchestrate multi-agent workflows. The tradeoff is cold starts on infrequently used functions, especially if your deployment package is large because you bundled a full SDK plus dependencies.

Cloudflare Workers is the fastest option for cold starts because it runs on V8 isolates instead of full containers, but it has a much tighter CPU-time limit and no native support for long-running background work unless you pair it with Durable Objects or Queues. Good fit for lightweight agents that make one or two tool calls per turn.

Modal and similar Python-native platforms (Fly.io Machines, Railway) are built with ML workloads in mind. They give you GPU access if your agent needs local inference alongside API calls, and they don't punish you as hard for long-running processes. The tradeoff is you're managing more infrastructure surface than a pure FaaS platform.

Vercel Functions work well if your agent lives behind a Next.js app already, and their newer Fluid Compute mode specifically targets long-running AI workloads by keeping instances warm and billing for actual CPU time rather than wall-clock time.

For most teams starting out, the right call is: use whatever platform your API layer already runs on, and only migrate to Modal or a container-based platform once you hit a hard limit (GPU need, execution time cap, or cost at scale).

Packaging an Agent for Serverless: The Core Pattern

Regardless of platform, the shape of an agent deployment is the same:

1. A handler function receives the incoming request (a new message, a webhook, a queue event).
2. It loads or initializes conversation state.
3. It runs the agent loop: call the model, execute any requested tools, feed results back, repeat until the model returns a final answer or you hit a step limit.
4. It persists updated state and returns the response.

Here's a minimal but complete version of that loop, independent of any particular platform:

import json

def run_agent_loop(client, model, messages, tools, tool_functions, max_steps=8):
    for step in range(max_steps):
        response = client.messages.create(
            model=model,
            max_tokens=1024,
            messages=messages,
            tools=tools,
        )

        if response.stop_reason != "tool_use":
            return response, messages

        messages.append({"role": "assistant", "content": response.content})

        tool_results = []
        for block in response.content:
            if block.type == "tool_use":
                fn = tool_functions[block.name]
                result = fn(**block.input)
                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": json.dumps(result),
                })

        messages.append({"role": "user", "content": tool_results})

    raise RuntimeError("Agent exceeded max_steps without finishing")

The max_steps guard matters more on serverless than anywhere else. Without it, a model stuck in a tool-calling loop will run until the platform kills it, and you pay for every one of those wasted invocations.

Example: Deploying a Claude-Powered Agent on AWS Lambda

Here's a working Lambda handler that wraps the loop above. It assumes a research agent with a single web-search-style tool and returns the final answer as JSON.

import os
import json
import anthropic

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
MODEL = os.environ.get("AGENT_MODEL", "claude-sonnet-4-5")

TOOLS = [
    {
        "name": "lookup_docs",
        "description": "Search internal documentation for a given query",
        "input_schema": {
            "type": "object",
            "properties": {"query": {"type": "string"}},
            "required": ["query"],
        },
    }
]

def lookup_docs(query):
    # Replace with a real search call (vector DB, Elasticsearch, etc.)
    return {"results": [f"Doc snippet related to: {query}"]}

TOOL_FUNCTIONS = {"lookup_docs": lookup_docs}

def run_agent_loop(messages, max_steps=8):
    for _ in range(max_steps):
        response = client.messages.create(
            model=MODEL,
            max_tokens=1024,
            messages=messages,
            tools=TOOLS,
        )
        if response.stop_reason != "tool_use":
            return response

        messages.append({"role": "assistant", "content": response.content})
        tool_results = []
        for block in response.content:
            if block.type == "tool_use":
                result = TOOL_FUNCTIONS[block.name](**block.input)
                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": json.dumps(result),
                })
        messages.append({"role": "user", "content": tool_results})

    raise RuntimeError("Agent exceeded max_steps")

def handler(event, context):
    body = json.loads(event.get("body") or "{}")
    user_message = body.get("message", "")

    messages = [{"role": "user", "content": user_message}]
    final_response = run_agent_loop(messages)

    text = "".join(
        block.text for block in final_response.content if block.type == "text"
    )

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps({"reply": text}),
    }

Deploy this with a requirements.txt containing anthropic, package it with a tool like sam build or a plain zip, and set ANTHROPIC_API_KEY and AGENT_MODEL as Lambda environment variables (never hardcode the key). Set the function timeout to at least 60 seconds, higher if your tools call slow external services. Memory allocation on Lambda also scales CPU, so 512MB to 1GB is a reasonable starting point for an agent that isn't doing local inference.

Handling Long-Running Agent Loops Without Timing Out

The single biggest source of production incidents in agent serverless deployment is timeout mismatch: the platform kills the function before the agent loop finishes, and the caller gets a 502 with no explanation.

Three approaches fix this, in order of how much complexity they add:

Raise the timeout and cap the step count. The simplest fix. Set your platform timeout close to its maximum (15 minutes on Lambda) and enforce a strict max_steps in your agent loop so you never actually approach that ceiling. This works fine until your traffic grows and you're paying for long-held invocations even on the rare slow request.

Move to async with polling or webhooks. Instead of blocking the HTTP response on the full agent loop, accept the request, kick off the agent run in the background (a queue-triggered Lambda, a Cloudflare Queue consumer, a Modal function), and let the client poll a status endpoint or receive a webhook when done. This decouples your API response time from the agent's actual runtime and is the right pattern for anything that might run past 30 seconds.

Stream partial results. For chat-style agents, stream tokens and tool-call status back to the client as they happen using Server-Sent Events or a WebSocket, rather than waiting for the whole loop. Lambda Function URLs support response streaming natively; Cloudflare Workers support it via ReadableStream. This is the best user experience but adds real implementation work on both the backend and the client.

For most production agents, the async pattern is the sweet spot: cheap to build, doesn't require the client to hold a long connection open, and survives traffic spikes without needing you to bump timeouts platform-wide.

State, Memory, and Tool Calls Across Invocations

Because serverless functions don't persist memory between invocations, conversation state has to live somewhere external. The two common choices:

• A fast key-value store (Redis, DynamoDB, Upstash) for active conversation state: the message history, any scratch variables the agent is tracking, and a TTL so abandoned sessions clean themselves up.
• A relational or document database for anything that needs to survive long-term or be queried later: full transcripts, tool-call audit logs, user preferences the agent should remember across sessions.

A simple pattern: store the running message array in Redis keyed by a session ID, load it at the start of the handler, append new turns, and write it back at the end. Keep the write inside the same handler invocation so a crash mid-loop doesn't leave state half-updated.

import redis
import json

r = redis.Redis(host=os.environ["REDIS_HOST"], decode_responses=True)

def load_session(session_id):
    raw = r.get(f"session:{session_id}")
    return json.loads(raw) if raw else []

def save_session(session_id, messages, ttl_seconds=3600):
    r.set(f"session:{session_id}", json.dumps(messages), ex=ttl_seconds)

Keep tool call results in state too, not just the model's text output. If a user asks a follow-up that references something a tool returned three turns ago, the model needs that in context, not just a summary of what it said.

Cold Starts and Latency Tuning

Cold starts hurt more with agents than with plain functions because you're already adding model latency on top. A few concrete levers:

• Keep the deployment package small. Don't bundle an entire ML framework if you're only calling an API. The anthropic SDK and a thin set of dependencies keep Lambda cold starts in the low hundreds of milliseconds range instead of several seconds.
• Use provisioned concurrency (Lambda) or keep-warm pings for agents on the critical path of a user-facing product, where a 2-second cold start is noticeable.
• Initialize clients outside the handler function, at module load time, so the SDK client and any connection pools are reused across warm invocations instead of rebuilt every call. The Lambda example above already does this with client = anthropic.Anthropic(...) at module scope.
• Separate latency-sensitive and latency-tolerant agents into different functions. A quick classification agent and a slow multi-step research agent shouldn't share a deployment; give the fast one tighter memory and timeout settings tuned for its actual workload.

Observability: Logging, Tracing, and Cost Tracking

Agent behavior is nondeterministic, so when something goes wrong in production you need to reconstruct exactly what happened: which tools were called, in what order, with what inputs, and what the model decided at each step.

Log the full trace of each agent run, not just the final output. At minimum, capture:

• The full message history for the run (redact anything sensitive before storing).
• Which tools were invoked, their inputs, and their outputs.
• Token usage per call, so you can attribute cost per agent run and per user.
• Total wall-clock time and step count, so you can spot runs approaching your max_steps ceiling before they start failing outright.

Wrap this in a decorator or middleware around your agent loop rather than sprinkling log statements through the tool-calling code:

import time
import logging

logger = logging.getLogger("agent")

def traced_run(session_id, run_fn, *args, **kwargs):
    start = time.time()
    try:
        result = run_fn(*args, **kwargs)
        logger.info(json.dumps({
            "session_id": session_id,
            "duration_ms": int((time.time() - start) * 1000),
            "status": "success",
        }))
        return result
    except Exception as exc:
        logger.error(json.dumps({
            "session_id": session_id,
            "duration_ms": int((time.time() - start) * 1000),
            "status": "error",
            "error": str(exc),
        }))
        raise

Ship these logs to whatever your platform's native sink is (CloudWatch, Cloudflare's logpush, Modal's built-in logs) and, if you have more than one agent in production, forward them to an OpenTelemetry collector so traces across agent, tool call, and downstream service line up in one view. Without this, debugging a bad agent run means guessing.

Security: Secrets, Sandboxing, and Least Privilege

Two mistakes show up repeatedly in agent serverless deployments:

Secrets baked into the deployment package instead of environment variables or a secrets manager. Never put an API key in code, even in a .env file that gets zipped up with the function. Use your platform's secrets manager (AWS Secrets Manager, Cloudflare secrets, Modal secrets) and inject at runtime.

Tools with unrestricted access given directly to the agent. If a tool can run arbitrary shell commands or query a production database with no row-level limits, the agent has that same access, and a bad prompt (or a compromised one, via prompt injection from untrusted tool output) can do damage. Scope every tool function down to the narrowest permission it needs: a read-only database role for lookup tools, a scoped API key for external calls, a sandboxed execution environment for anything involving code execution.

If your agent executes code as one of its tools, run that code in an isolated environment separate from the function that's orchestrating the agent loop, not inside the same Lambda invocation. A dedicated sandboxing service or a short-lived container keeps a runaway or malicious code execution from touching your orchestration layer's credentials.

A Deployment Checklist Before You Ship

Before pointing real traffic at an agent running on serverless infrastructure, confirm:

• Timeout headroom. Your platform timeout comfortably exceeds your agent's worst-case max_steps runtime, or you've moved to an async/polling pattern.
• Step limit enforced. The agent loop cannot run indefinitely; there's a hard cap and a clear error path when it's hit.
• State externalized. Conversation history and tool results live in a store outside the function, not in memory.
• Secrets in a secrets manager. No API keys in code or in the deployment package.
• Tool permissions scoped. Every tool has the minimum access it needs, nothing more.
• Full run tracing. You can reconstruct any agent run from logs: tool calls, inputs, outputs, token counts, duration.
• Cost per run visible. You're tracking token usage well enough to know what a single agent invocation costs, not just aggregate monthly spend.
• Cold start tested. You've measured actual cold start latency for your deployment package size, not assumed it based on platform defaults.

FAQ

What's the difference between deploying a regular API and deploying an AI agent on serverless? A regular API endpoint does bounded, predictable work per request. An agent runs a loop of model calls and tool calls with a variable number of steps and variable duration, so you need to design around timeouts, step limits, and external state in a way a typical CRUD endpoint never requires.

Which serverless platform is best for AI agents? There's no universal answer. AWS Lambda is the safest default if you need long execution windows and already run on AWS. Cloudflare Workers is best for fast, lightweight agents with few tool calls. Modal and similar platforms suit agents that need GPU access or run for extended periods without punishing cold-start behavior.

How do I stop an agent from running forever on serverless? Enforce a max_steps limit in your agent loop, independent of the platform's own timeout. This gives you a controlled failure (a clear error you can log and retry) instead of an uncontrolled one (the platform killing the function mid-execution with no cleanup).

Do I need a database for a serverless agent, or can I keep everything stateless? You need external state for anything beyond a single-turn, no-memory interaction. Even a simple follow-up question requires the agent to see the prior conversation, and serverless functions don't retain memory between invocations, so a key-value store like Redis or DynamoDB is close to mandatory for multi-turn agents.

How do I keep serverless agent costs under control? Track token usage per run, not just per month, so you can see which conversations or tool patterns are expensive. Cap max_steps so a stuck loop doesn't burn through model calls, cache tool results where the same lookup is likely to repeat, and choose the smallest model that reliably handles each task rather than defaulting every agent to your most capable one.

Can I stream responses from a serverless AI agent? Yes, though support varies by platform. AWS Lambda supports streaming through Function URLs, Cloudflare Workers support it via the Streams API, and most modern serverless platforms have added first-class support for this because AI workloads made it a common requirement. Streaming adds implementation complexity on both server and client but meaningfully improves perceived latency for chat-style agents.