Prompt Caching Explained

Prompt caching lets a large language model reuse the processed state of a prompt instead of recomputing it on every request, and it is one of the highest-leverage optimizations you can make to an LLM-powered application. If you send the same system prompt, tool definitions, or reference document on every call, prompt caching can cut input costs by roughly 90 percent and cut time-to-first-token dramatically. The catch is that prompt caching is a strict prefix match: change a single byte anywhere before your cache marker and the whole cache entry is void, silently, with no error. This guide covers how prompt caching actually works under the hood, the TTL options, the pricing multipliers, the minimum token thresholds, and exactly where to place your cache_control breakpoints so you are not paying for a feature you accidentally disabled.

What Prompt Caching Actually Does

When you send a request to an LLM API, the model has to process every token in your prompt before it can generate a single token of output. That processing (loading the tokens into the model's internal representation) is the expensive, slow part for long prompts. Prompt caching lets the API save that intermediate state after processing a prompt once, keyed to the exact sequence of bytes it processed. The next time you send a request that starts with those same bytes, the API skips reprocessing that portion and picks up right where the cache left off.

This only works because of one property: prompt caching is a prefix match, not a fuzzy match, not a semantic match, and not a per-message match. The cache key is derived from the exact rendered bytes of your prompt up to a marker you place. If your system prompt is 5,000 tokens and your user question is 20 tokens, and the system prompt is byte-identical to the last request, the API can cache all 5,000 tokens and only bill you full price for the 20-token question.

The practical implication is that prompt caching rewards prompts with a large, stable prefix and a small, variable suffix. A customer support bot with a long knowledge base baked into the system prompt is a great fit. A one-off summarization call with a different document every time is not, because there is no repeated prefix to cache.

Render Order: Tools, Then System, Then Messages

Before you can place a breakpoint correctly, you need to know how the API assembles your request internally. The render order is fixed: tools first, then system, then messages. This matters because a cache breakpoint on the last block of your system prompt caches everything that came before it too, meaning your tool definitions get cached along with the system text, as long as neither changed.

This is also why changing your tool list invalidates more than you might expect. Adding, removing, or reordering a single tool shifts the byte sequence at the very start of the prompt, which cascades forward and invalidates every cache entry downstream of it, including the system prompt and the conversation history. Tools are the most upstream thing you control, so they need to be the most stable.

Setting Up cache_control in Your Requests

Prompt caching is controlled with a cache_control field. There are two ways to use it: an automatic top-level setting that caches the last cacheable block for you, or manual placement on specific content blocks for fine-grained control.

Automatic caching is the simplest option when you do not need precise placement:

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},
    system="You are an expert on this large reference document...",
    messages=[{"role": "user", "content": "Summarize the key points"}],
)

Manual placement gives you control over exactly which block gets the marker, which matters once your prompt has multiple sections with different stability profiles:

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "You are an expert on this large reference document...",
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": "Summarize the key points"}],
)

The same pattern applies in TypeScript with camelCase-free JSON field names (the wire format is identical), and in raw HTTP you set the same cache_control object inside the relevant content block of the JSON body you POST to the messages endpoint. cache_control can be attached to system text blocks, tool definitions, or any message content block: text, image, tool_use, tool_result, or document blocks.

You are allowed a maximum of 4 cache_control breakpoints per request. That is not a lot, so use them at genuine stability boundaries rather than sprinkling them everywhere.

Minimum Token Requirements

Prompt caching will not activate on very short prompts, even if you set cache_control correctly. There is a minimum cacheable prefix length, and it varies by model tier. If your prefix is shorter than the minimum, the request runs normally with no error, but you will see zero cache activity in the usage stats.

The minimums, from the more capable model tiers down:

• Higher-tier models (the Opus-class and equivalent premium tiers): 4096 tokens minimum
• Mid-tier and some smaller models: 2048 tokens minimum
• Some lighter-weight model families: 1024 tokens minimum

This means a 3,000-token system prompt might cache reliably on a smaller or mid-tier model but silently fail to cache at all on a larger flagship model that requires 4096 tokens as the floor. If you are testing prompt caching and seeing no cache hits, check your prefix length against your specific model's minimum before assuming something else is broken.

TTL: 5-Minute vs 1-Hour Cache Lifetimes

A cached prompt does not live forever. It has a time-to-live (TTL), and you choose between two options:

The default is a 5-minute ephemeral TTL. Every time the cache is read, the 5-minute clock resets, so a cache entry that gets hit regularly can stay warm indefinitely. If 5 minutes pass with no read, the entry expires and the next request pays full price to rebuild it.

You can opt into a 1-hour TTL instead, which is useful for workloads with bursty traffic and long gaps between bursts, where the default 5-minute window would expire between bursts and force a full rebuild every time.

system=[
    {
        "type": "text",
        "text": "You are an expert on this large reference document...",
        "cache_control": {"type": "ephemeral", "ttl": "1h"},
    }
]

Choosing the right TTL is really a cost decision, which brings us to pricing.

Pricing Multipliers and Break-Even Math

Prompt caching changes what you pay for input tokens in three distinct ways, and understanding the multipliers is what tells you whether caching actually saves you money for a given traffic pattern.

Cache writes (the first time a prefix is processed and stored) cost more than a normal, uncached request:

• 5-minute TTL write: 1.25x the normal input token price
• 1-hour TTL write: 2x the normal input token price

Cache reads (every subsequent request that hits the same prefix) cost far less than a normal request:

• Cache read: roughly 0.1x the normal input token price, about a 90 percent discount

Tokens that are neither written nor read, meaning the part of your prompt outside any cached prefix, bill at the normal, full price.

This gives you a break-even calculation. With the default 5-minute TTL, you break even after just two requests: one write at 1.25x plus one read at 0.1x totals 1.35x, versus 2x for two uncached requests. Every request after that is pure savings. With the 1-hour TTL, the doubled write cost means you need at least three requests to break even: one write at 2x plus two reads at 0.2x totals 2.2x, versus 3x for three uncached requests.

The practical rule: if a prompt prefix will be reused at least twice within 5 minutes, cache it with the default TTL. If it will be reused across longer gaps but still multiple times, the 1-hour TTL usually still wins once you clear three or more hits. If a prompt is genuinely one-shot, do not add cache_control at all. It will pay the write premium and never earn back the discount.

Where to Place Breakpoints

Placement is the single biggest lever in prompt caching, more than any other setting. Here are the patterns that come up most often.

Large, shared system prompt. Put one breakpoint on the last text block of your system prompt. Because tools render before system, this single marker captures tools and system together in one cached unit.

system=[
    {
        "type": "text",
        "text": "<large shared instructions and reference material>",
        "cache_control": {"type": "ephemeral"},
    }
]

Multi-turn conversations. Put the breakpoint on the last content block of the most recently appended turn, every time you send a new request. Each new request then reuses the entire prior conversation as a cached prefix, and the discount compounds as the conversation grows longer, because earlier breakpoints remain valid read points.

Shared prefix with a varying suffix. If many requests share a large fixed block, like few-shot examples or a retrieved document, but each ends with a different question, place the breakpoint at the end of the shared portion only, not at the end of the whole prompt. If you put the marker after the varying suffix, every request writes a unique cache entry that is never read again, and you pay the write premium for nothing.

messages = [{
    "role": "user",
    "content": [
        {
            "type": "text",
            "text": "<shared context, examples, or retrieved documents>",
            "cache_control": {"type": "ephemeral"},
        },
        {"type": "text", "text": "<the varying question, no marker>"},
    ],
}]

Prompts that differ from the very first token. If there is no meaningful shared prefix at all, do not cache. Adding a marker to a prompt with nothing stable in it only adds the write premium with zero chance of a read.

Common Reasons Cache Hits Silently Drop to Zero

Because prompt caching fails silently (no error, just a cache miss), the debugging process is really an audit of what changes between requests that should otherwise be identical. The usual suspects, in order of how often they show up in real codebases:

A timestamp or current date interpolated into the system prompt. Something like inserting "current date: 2026-07-09" at the top of your instructions changes the prefix on every single request, which invalidates everything after it.

A random ID generated per request, such as a UUID or a request trace ID, placed early in the prompt content rather than at the very end after your last breakpoint.

Non-deterministic JSON serialization. If you build part of your prompt by serializing a dictionary or a set without sorting keys, the byte order can differ between otherwise-identical requests even though the logical content is the same.

A tool list that varies by user, feature flag, or session. Since tools render first, any variation there invalidates the entire downstream cache for every user who does not share the exact same tool configuration.

Conditional sections in the system prompt built with string concatenation based on flags, where every combination of flags produces a distinct prefix that never accumulates enough hits to pay off.

Switching models mid-session. Caches are scoped per model, so alternating between two models on the same conversation guarantees a cache miss every time you switch back.

The fix in every case is the same: move the volatile piece as far downstream as possible, ideally after your last cache_control marker, make serialization deterministic, and keep the tool list and model fixed for the lifetime of a cached conversation.

Verifying Cache Hits in the Response

Do not assume caching is working just because you added cache_control. Check the usage object on every response. It reports three relevant fields:

cache_creation_input_tokens tells you how many tokens were written to the cache on this request, meaning you paid the write premium.

cache_read_input_tokens tells you how many tokens were served from the cache on this request, meaning you paid the discounted read rate.

input_tokens is the uncached remainder only, billed at full price.

response = client.messages.create(...)
print(response.usage.cache_creation_input_tokens)
print(response.usage.cache_read_input_tokens)
print(response.usage.input_tokens)

If cache_read_input_tokens stays at zero across repeated requests that you believe share an identical prefix, something in the prefix is changing between calls. The fastest way to find it is to log the exact rendered prompt bytes for two consecutive requests and diff them directly, rather than guessing from the code.

One more subtlety worth knowing: total prompt size is the sum of all three fields, not just input_tokens. If you are running a long agentic loop and input_tokens looks small, that does not mean your prompt is small. It likely means most of it is being served from cache, and you should look at the sum to understand the real size.

Concurrent Requests and the Lookback Window

Two behaviors trip people up once they move past a single-request setup.

First, a cache entry only becomes readable after the first response begins streaming back. If you fire ten identical requests in parallel, none of them can read a cache the others are still writing, and all ten pay full price. The fix for fan-out patterns is to send one request first, wait for the first streamed token, then fire the remaining requests, which can now read what the first one just wrote.

Second, each breakpoint only looks back a limited number of content blocks (roughly 20) to find a prior cache entry. In an agentic loop that appends many tool_use and tool_result blocks in a single turn, a turn that adds more than that many blocks can push the next breakpoint outside the lookback window, causing a silent miss even though the content is technically still there. The fix is to place an intermediate breakpoint partway through long turns rather than only at the very end.

FAQ

Does prompt caching change the model's output? No. Caching only affects how the API processes and bills the input tokens it has already seen before. It has no effect on what the model generates, and it does not make responses more or less deterministic.

What happens if my prefix changes by even one character? Everything after that point in the prefix is treated as a fresh request. The API does not do partial or fuzzy matching, so a single changed character invalidates the entire downstream cache for that request. Content before the change point, if it has its own earlier breakpoint, can still be cached separately.

Is prompt caching worth it for a single one-off request? No. A cache write costs more than an uncached request (1.25x for the 5-minute TTL, 2x for the 1-hour TTL), so a prompt you only send once always loses money if you cache it. Caching pays off starting on the second request that hits the same prefix.

Can I cache images or tool definitions, not just text? Yes. cache_control can be attached to any content block type, including image blocks, document blocks, tool_use and tool_result blocks, and tool definitions themselves, not only plain text blocks.

Why is my cache_read_input_tokens always zero even though I added cache_control? The most common causes are a timestamp or random ID inserted into the prompt before your breakpoint, non-deterministic JSON key ordering, a tool list or model that varies between requests, or a prefix shorter than your model's minimum cacheable token threshold. Diff the exact rendered prompt bytes between two requests to isolate which one applies.

How many cache breakpoints can I use in one request? A maximum of four cache_control markers per request. Use them at genuine stability boundaries, such as the end of a shared system prompt, the end of shared few-shot examples, and the end of the growing conversation history, rather than adding one to every block.

Should I always use the 1-hour TTL instead of the default? Only if your traffic has gaps longer than 5 minutes between reuses of the same prefix but you still expect at least three total hits. The 1-hour TTL costs more to write (2x versus 1.25x), so for tight, frequent traffic the default 5-minute window is cheaper and resets on every read anyway.

Does switching models mid-conversation break the cache? Yes. Cache entries are scoped to a specific model, so any model switch forces a full, uncached rebuild on the next request, even if every other part of the prompt is identical.