How to Batch Requests in MCP Without Blowing Up Your Latency Budget

If you are building an agent on top of the Model Context Protocol, you will eventually hit a wall where every tool call goes out one at a time and the agent waits on each response before sending the next. MCP batch requests solve this by letting a client send an array of JSON-RPC requests in a single HTTP call or a single stdio write, so the server can process them together and the round trip only happens once. This matters most when your agent needs to call three or four independent tools before it can reason about the next step, because sequential calls turn a 200ms operation into a 2 second one.

This article walks through what batching actually does in MCP, how to implement it on both the client and server side, where it breaks down, and what to use instead when the spec or your transport does not support it well.

What MCP Batch Requests Actually Are

MCP is built on JSON-RPC 2.0, and JSON-RPC has always supported batching as part of the base spec: instead of sending one request object, you send an array of request objects, and the server responds with an array of response objects in whatever order it finishes them. In MCP terms, this means a client can bundle multiple tools/call, resources/read, or prompts/get requests into one payload instead of opening a new request for each one.

The appeal is straightforward. If your agent decides it needs the output of get_weather, get_traffic, and get_calendar_events before it can plan a route, sending three separate requests means three round trips, three lots of TCP or HTTP overhead, and three lots of server dispatch latency stacked in sequence unless you're already parallelizing them at the transport layer. A batch request collapses that into one write and one read on the wire, even though the server may still execute the underlying tool handlers concurrently or sequentially internally.

It is worth being precise about what batching buys you. It does not make the underlying tool calls faster. Reading a database is still going to take however long it takes. What batching removes is the network round-trip tax between the client and the MCP server. On a local stdio transport that tax is close to zero, so batching barely matters. On a remote HTTP-based MCP server, especially one behind a load balancer or with TLS handshake overhead on cold connections, batching can meaningfully cut wall-clock time.

The Two Places Batching Shows Up in MCP

There are two distinct kinds of batching people mean when they say "batch requests in MCP," and conflating them causes confusion.

The first is JSON-RPC-level batching: sending an array of request objects as the HTTP body or the stdio message, per the JSON-RPC 2.0 batch spec. This is transport-level plumbing and has nothing to do with what the tools do.

The second is tool-level batching: designing a single MCP tool that accepts an array of inputs and processes them together, for example a search_documents tool that takes a list of queries instead of one query. This is an API design choice you make when you write the server, independent of whether the transport itself supports JSON-RPC batching.

In practice, tool-level batching is more reliable and more widely supported today, because it does not depend on every client and server in the chain correctly implementing JSON-RPC batch semantics. If you control the server, adding a batch-shaped tool is often the pragmatic move even if the underlying MCP SDK's transport batching is inconsistent across versions.

Implementing JSON-RPC Batching on the Client

If your MCP client talks HTTP to the server (the Streamable HTTP transport), you can construct a batch by sending a JSON array instead of a single object as the request body. Here is a minimal example using plain fetch against a hypothetical MCP HTTP endpoint:

async function batchCall(endpoint, calls) {
  const body = calls.map((call, i) => ({
    jsonrpc: "2.0",
    id: i,
    method: "tools/call",
    params: {
      name: call.tool,
      arguments: call.args,
    },
  }));

  const res = await fetch(endpoint, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Accept: "application/json, text/event-stream",
    },
    body: JSON.stringify(body),
  });

  const results = await res.json();
  // results is an array; match back to calls by id
  return results.sort((a, b) => a.id - b.id);
}

const results = await batchCall("https://mcp.example.com/rpc", [
  { tool: "get_weather", args: { city: "Bangalore" } },
  { tool: "get_traffic", args: { city: "Bangalore" } },
  { tool: "get_calendar_events", args: { date: "today" } },
]);

The key details: every request in the array needs a unique id so you can match responses back to requests, since the server is allowed to return them out of order. The Accept header needs to list both application/json and text/event-stream because Streamable HTTP servers can respond either way depending on whether they stream. And you should always handle the case where the server returns a single object instead of an array, because not every MCP server implementation honors batch input even if the client sends it.

If you are using an official SDK (the TypeScript or Python MCP SDK), check whether the client class exposes a batch method directly rather than hand-rolling the JSON-RPC array yourself. SDK support for batching has been inconsistent across versions because the MCP spec itself made batch support optional and some transport revisions have deprecated it in favor of other patterns. Read your SDK's changelog before assuming batch semantics will just work.

Implementing Batch Handling on the Server

On the server side, if you are hand-rolling an MCP server without a framework, you need to detect whether the incoming payload is an array and, if so, process each entry and return an array of responses in the same shape.

async function handleRequest(req, res) {
  const payload = req.body;
  const isBatch = Array.isArray(payload);
  const requests = isBatch ? payload : [payload];

  const responses = await Promise.all(
    requests.map(async (rpcReq) => {
      try {
        const result = await dispatchToolCall(rpcReq.method, rpcReq.params);
        return { jsonrpc: "2.0", id: rpcReq.id, result };
      } catch (err) {
        return {
          jsonrpc: "2.0",
          id: rpcReq.id,
          error: { code: -32000, message: err.message },
        };
      }
    })
  );

  res.json(isBatch ? responses : responses[0]);
}

Two things matter here. First, use Promise.all (or your language's equivalent concurrent-await pattern) so the individual tool calls actually run concurrently rather than one after another inside the loop, otherwise batching the request buys you nothing on the server side even though it saved a round trip on the wire. Second, keep each entry's error isolated: one failing tool call inside a batch should not take down the other results, which is why each entry gets its own try/catch and returns its own JSON-RPC error object rather than failing the whole response.

If you are building your server with the official MCP SDK instead of hand-rolling JSON-RPC, check whether the SDK's transport layer already handles batch parsing for you. Most recent SDK versions do, and duplicating that logic yourself is wasted effort and a source of subtle bugs around request ID matching.

Tool-Level Batching: The More Durable Pattern

Because transport-level batch support has been a moving target in the MCP spec, many production MCP servers sidestep the question entirely by designing tools that accept arrays natively. Instead of exposing get_user(id) and hoping the client batches ten calls to it, you expose get_users(ids) and let the tool handler do the batching internally, often against a single SQL IN clause or a single upstream batch API call.

{
  "name": "get_users",
  "description": "Fetch multiple users by ID in a single call",
  "inputSchema": {
    "type": "object",
    "properties": {
      "ids": {
        "type": "array",
        "items": { "type": "string" },
        "maxItems": 50
      }
    },
    "required": ["ids"]
  }
}

The server-side handler then does one query instead of N:

async function getUsers({ ids }) {
  const rows = await db.query(
    "SELECT id, name, email FROM users WHERE id = ANY($1)",
    [ids]
  );
  return { content: [{ type: "text", text: JSON.stringify(rows) }] };
}

This pattern has three advantages over relying on JSON-RPC batching. It works identically on stdio and HTTP transports, since there is nothing transport-specific about it. It lets you push a real batch optimization down to the data layer, like a single database round trip instead of N separate queries even if the LLM client happened to bundle its JSON-RPC calls. And it composes better with how LLMs actually call tools, since a well-described array parameter is something the model can reason about and populate directly from context, whereas relying on the client runtime to auto-batch several separate tool calls depends on orchestration logic the model has no visibility into.

The tradeoff is that you have to design for it up front. If your tool schema only accepts a single ID, retrofitting batch support later means adding a new tool or a breaking change to the existing one, so it is worth deciding early whether a given tool is likely to be called many times in a row for the same operation.

Where Batching Breaks Down

A few failure modes come up often enough to call out directly.

Not every MCP server respects batch input even when the transport nominally supports it. Some implementations will 400 on an array body, others will silently only process the first element. Always test against the actual server you are integrating with rather than assuming batch support because the spec allows it.

Streaming responses and batching do not mix cleanly. If a tool call is expected to stream partial results back (for example a long-running search), bundling it into a batch with other calls that return immediately creates ordering and buffering problems, because the client has to decide whether to wait for the whole batch or process results as they arrive. Most implementations punt on this by disallowing streaming tools inside a batch.

Error handling gets easy to get wrong. If one call in a batch of five fails, does the whole batch fail, or do you get four successes and one error object? The JSON-RPC spec says each request in a batch is independent and gets its own response, but plenty of hand-rolled servers treat the batch as all-or-nothing. Confirm this behavior before you write client code that assumes partial success.

Rate limiting and quota enforcement can get confused by batches. If your MCP server enforces per-minute call limits by counting inbound HTTP requests, a client that sends one giant batch of fifty tool calls per request will slip past a limiter designed to count requests rather than count the tool calls inside them. If you run an MCP server behind a rate limiter, make sure it counts the array length, not the request count.

Finally, very large batches can hit body-size limits before they hit any MCP-specific limit. A batch of a few hundred tool calls with verbose arguments can easily exceed a typical reverse proxy's request body cap. Chunk large batches client-side into groups of a few dozen rather than sending everything as one payload.

A Practical Rule of Thumb

Reach for JSON-RPC batching when you know at agent-construction time that you need several independent, non-streaming tool results before you can proceed, and when you control both ends of the transport well enough to have verified the server actually honors batch arrays. This is a good fit for orchestration code you write yourself, not for arbitrary LLM tool-calling loops, because the decision to batch calls together is usually made by your agent's planning logic, not by the model picking tools one at a time.

Reach for tool-level array parameters when the batching decision naturally belongs to the tool itself, such as fetching multiple records, running multiple independent computations, or checking multiple resources of the same type. This is the more robust default because it does not depend on transport-level features that vary across MCP SDK versions and server implementations.

And if you are still calling tools one at a time inside a loop and each call is cheap and independent, check whether your agent framework already parallelizes tool calls that do not depend on each other's output before you reach for batching at the protocol level. Sometimes the actual latency problem is that your orchestration code is awaiting calls sequentially, not that the transport lacks a batch endpoint.

FAQ

Does MCP support batch requests out of the box? JSON-RPC 2.0, which MCP is built on, supports batching as part of its base specification, so an array of request objects is valid input at the protocol level. Whether a specific MCP client SDK or server implementation actually exposes and honors that array format has varied across spec revisions and SDK versions, so verify support against the exact server and SDK versions you are using rather than assuming it from the base JSON-RPC spec alone.

What is the difference between batching and parallel tool calls? Batching is about how many requests go out on the wire in a single write, typically one JSON array in one HTTP call. Parallel tool calls are about whether the server (or client orchestration code) executes the underlying handlers concurrently rather than one after another. You can batch requests on the wire but still process them sequentially on the server, and you can also send requests one at a time on the wire while still running them concurrently server-side if your client fires them without waiting on each response.

Should every MCP tool accept an array of inputs? No. Tools that are naturally single-item operations, like creating one record or sending one message, should stay single-item to keep the schema simple and the model's tool-calling decisions unambiguous. Add an array-accepting variant only for tools that are commonly called many times in a row for the same operation, such as fetching multiple records by ID or running the same check against several resources.

Can I batch calls to different tools in one request? With JSON-RPC-level batching, yes: each entry in the array is its own independent request and can target a different tool, method, or even a different type of MCP request such as a mix of tools/call and resources/read. With tool-level batching, no: a single tool's array parameter only batches calls to that one tool, so mixing different operations requires either JSON-RPC batching or separate requests.

Does batching help if my MCP server and client are both local over stdio? Barely. The stdio transport does not have TCP handshake or TLS overhead, so the round-trip cost per message is already small. Batching mostly pays off on HTTP-based transports, especially remote servers with real network latency, connection setup costs, or per-request authentication overhead. For local stdio servers, focus on server-side concurrency instead of client-side batching.

What happens if one call in a batch fails? Per the JSON-RPC spec, each request in a batch produces its own independent response, so a failure in one call should return as a JSON-RPC error object at that array position while the other calls still return their successful results. Not every server implementation follows this correctly, so test the failure path explicitly (send a batch with one intentionally invalid call) before relying on partial-success behavior in production.