Prompt Caching: How Anthropic and OpenAI Differ (and the Catch)

June 23, 2026 · updated June 25, 2026 · The Hidden Cost of AI Coding (part 5)

▶ Watch on YouTube & subscribe to The Stack Underflow

In a multi-step AI agent, the model has no memory between calls. Every step has to resend system instructions, tool schemas, and the full conversation history — only the newest user message is actually new. The earlier episode in this series put a number on the waste: 62% of a typical agent’s input bill is resent context. Prompt caching is the direct fix: pay to process a stable block of tokens once, then reuse that work on every subsequent call at a fraction of the cost.

Most teams never configure it, leaving 25–50% of their input bill untouched. This tutorial walks through how caching works on both Anthropic and OpenAI, what the break-even math looks like, and the single most common mistake that silently destroys your hit rate.

The one-sentence version: Prompt caching lets you pay once to process a stable, repeated prefix and reuse that work on every subsequent call — but only if your prompts are structured so the stable content comes first.

Why the same tokens keep getting billed

Consider a three-step agent run:

Step 1:  [System prompt] + [Tool schemas] + [User message 1]
Step 2:  [System prompt] + [Tool schemas] + [History so far] + [User message 2]
Step 3:  [System prompt] + [Tool schemas] + [History so far] + [User message 3]

Steps 2 and 3 resend almost everything from step 1. Without caching, you pay full price to re-tokenize and re-process that identical block each time. Three steps, three identical bills for the same content.

Prompt caching breaks that pattern. The first time the model sees the large stable block, it does the work and saves the intermediate computation result — called the KV cache. Every subsequent request that begins with the same token sequence reuses the saved result instead of recomputing it. Same answer quality, dramatically less compute, lower latency.

The KV cache stores attention keys and values, not output text. This is an important distinction: you are not replaying a saved answer, you are skipping the re-computation of an already-processed prefix. The model reasons over the actual input every time; it just does not redo the prefix arithmetic.

Two flavors: Anthropic vs. OpenAI

The two major providers implement caching differently, and the trade-offs matter for how you design your prompts.

Anthropic: explicit caching with cache_control

On Anthropic models, nothing is cached unless you opt in. You mark the end of the stable prefix with a cache_control header on the relevant message or content block. Anthropic then splits the cacheable portion from the dynamic tail and stores the KV cache for the marked prefix.

The current Claude family (Claude Sonnet 4.6, Claude Haiku 4.5, Claude Opus 4.8) all support prompt caching, with a minimum cacheable prefix that varies by model — generally 1,024 tokens for Haiku-tier models and up to 4,096 tokens for Opus-tier models. Check the latest docs for the specific minimum for the model you are using. (docs.anthropic.com, 2025)

Here is what the cache_control marker looks like in a typical API call:

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "You are a helpful assistant. [10,000 tokens of stable instructions...]",
            "cache_control": {"type": "ephemeral"}
        }
    ],
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

# Check cache behavior in the usage object
usage = response.usage
print(usage.cache_creation_input_tokens)  # tokens written to cache this call
print(usage.cache_read_input_tokens)      # tokens served from cache this call
print(usage.input_tokens)                 # tokens billed at full price

The economics require some thought. A cache write costs 1.25x the normal input price for the 5-minute TTL. A cache read costs 0.1x. If you write a cache entry and read it exactly once in the same 5-minute window, you pay 1.25x + 0.1x = 1.35x versus 2x for two uncached calls — you are ahead, but only barely. The break-even is roughly 1.5 reads per write. Write once, never read, and you pay a net premium over skipping caching entirely.

The default TTL is 5 minutes. An extended 1-hour TTL is available at a 2x write cost (versus 1.25x for the 5-minute version). If your agent runs are spaced further apart than the active TTL, your hit rate collapses and you pay the write premium with none of the read savings. Always plan around your real call frequency before selecting a TTL tier.

OpenAI: automatic caching

OpenAI caches transparently. Any prefix longer than 1,024 tokens is eligible — no markers, no structural changes needed. Reads cost 50% off normal input price; there is no extra write charge. The trade-off is less control: you cannot target exactly what gets cached, and you cannot force a cache flush.

The 50% discount is smaller than Anthropic’s 90%, but the zero write cost and zero configuration make it simpler to capture savings without restructuring prompts. For workloads where you cannot guarantee call frequency — batch processing, low-traffic agents, or development environments — OpenAI’s automatic caching is more forgiving of irregular access patterns.

The one mistake that kills your hit rate

Caching only works if the prefix is byte-for-byte identical on every call. Change even a single token at the start of the stable block and the cache misses — even if the next 10,000 tokens are identical.

The classic trap: embedding a timestamp, request ID, or user identifier near the top of the system prompt.

# BAD — timestamp in the prefix kills cache hits
system_prompt = f"""
Current time: {datetime.utcnow().isoformat()}
You are a helpful assistant...
[10,000 more tokens of stable content]
"""

Every call generates a unique prefix. Zero cache hits. You pay the write cost every time with no reads to offset it.

The fix is a simple structural rule:

GOOD — stable content first, variable content last

1. System prompt (static instructions)       ← cache this block
2. Tool schemas (rarely change)              ← cache this block
3. Conversation history (grows, but stable
   for this specific call)                   ← can cache if stable
4. Current user message (only new thing)     ← never in the cached prefix

Put anything dynamic — timestamps, user IDs, session state, the current date — at the very end of the context. The prefix the model reads first must be identical across calls for cache hits to accumulate.

Context window with good cache structure

┌─────────────────────────────────────────────────┐
│  [STABLE — mark with cache_control]              │
│  System prompt: 2,000 tokens                    │
│  Tool schemas: 5,000 tokens                     │
│  ─────────────────────────────────────────────  │
│  [DYNAMIC — never cached]                        │
│  User message: 50 tokens                        │
│  Current time: 2026-06-25T10:00:00Z             │
└─────────────────────────────────────────────────┘

Cache reads the stable block on every call after the first.
The dynamic tail does not break it.

Real numbers

A 10,000-token prefix on Claude Sonnet 4.6 at the uncached input price of $3.00 per million tokens costs roughly $0.03 per call.

The Anthropic read discount is steep enough that even a modest volume of repeated calls saves substantially. At scale, connecting back to the earlier episode’s 62% resent-context finding: prompt caching done well can knock 30–50% off your total input bill with zero change in output quality. On a $1,000/month spend, that is $300–$500 back for one afternoon of prompt restructuring.

The numbers above use Claude Sonnet 4.6. Claude Haiku 4.5 (at $1.00/$5.00 per MTok input/output) produces proportionally smaller absolute savings but the same relative percentages. Claude Opus 4.8 (at $5.00/$25.00 per MTok) makes caching even more valuable per call.

Three things that make caching pay off

1. Stable prefix at the top. Variable tokens near the start destroy hit rates. Audit your system prompt for anything that changes per-call and move it to the end.

2. Enough reads per cache write. For Anthropic with the 5-minute TTL, you need at least 1.5 reads per write to break even — and ideally far more to justify the overhead. High-frequency agents (multiple calls per minute) benefit most. Low-frequency or batch workloads may fare better with OpenAI’s zero-write-cost approach.

3. Measurement. Check the cache_read_input_tokens and cache_creation_input_tokens fields in Anthropic API responses before assuming caching is working. If cache_read_input_tokens is zero across repeated calls with the same prefix, your prefix is not stable. This is the most reliable diagnostic — the numbers do not lie.

Common misconceptions

“Caching changes the model’s output.” It does not. The KV cache stores intermediate computation results — attention keys and values — not final answers. Output quality is identical to a fully uncached call. This is verifiable: run the same prompt with and without a warm cache and compare outputs.

“Caching is automatic on Anthropic.” Only on OpenAI. Anthropic requires explicit cache_control markers on the content blocks you want cached. If you do not add them, nothing is cached regardless of prefix length. Teams migrating from OpenAI often miss this and wonder why their Anthropic costs are higher than expected.

“The cache persists indefinitely.” Anthropic’s default TTL is 5 minutes. An extended 1-hour TTL is available at a higher write rate. If your use pattern has long gaps between calls — overnight batch jobs, infrequent user sessions — you will pay the write premium repeatedly without accumulating reads. Plan your TTL tier around your real call frequency, not a theoretical maximum.

“As long as the content is the same, it will cache.” Token order matters too. Identical text structured in a different order produces different tokens and misses the cache. Even whitespace differences or encoding changes can break prefix identity. Treat the cached prefix as an immutable artifact: once it works, do not touch it.

“The 1-hour TTL is always better than 5 minutes.” The 1-hour TTL costs 2x the write price instead of 1.25x. If your calls are clustered within short windows, the 5-minute TTL is more economical. The 1-hour TTL only wins if reads are spread over longer periods than 5 minutes and you have enough read volume to overcome the higher write cost.

Frequently asked questions

Does prompt caching work across different users or sessions?

Yes, within your API key scope. Cache entries are tied to your API key, not to individual users. Any call from any user in your system that starts with the same stable prefix will hit the same cache entry — which is precisely why keeping the system prompt and tool schemas identical across users multiplies the savings. The cache is not shared globally between organizations.

What happens if I update my system prompt mid-deployment?

The old cache entry is invalidated and a new one is written at the write cost. For Anthropic, design your system prompt to be stable for at least the TTL window — ideally for the full deployment period between intentional updates. For OpenAI, updates propagate automatically with no extra penalty beyond the first uncached call. If you deploy frequently (CI/CD pipelines, A/B tests), factor the write cost overhead into your cost model.

Should I use Anthropic or OpenAI caching?

It depends on your call pattern. Anthropic’s 90% read discount wins at high call frequency — once you clear the 1.5-read break-even, every additional read is very cheap. OpenAI’s zero write cost wins for infrequent or unpredictable workloads where you might not accumulate enough reads to break even on the write premium. Run the break-even math against your actual call distribution before choosing. If you are already committed to one provider for other reasons, use that provider’s caching rather than switching.

How do I know if caching is actually working?

Anthropic returns cache_creation_input_tokens (tokens written to cache) and cache_read_input_tokens (tokens served from cache) in every API response’s usage object. OpenAI surfaces a cached_tokens field in the usage object. If cache_read_input_tokens is zero across repeated calls with the same prefix, your prefix is not stable — audit the prompt for per-call variables. A healthy cache will show cache_creation_input_tokens on only the first call and cache_read_input_tokens on all subsequent ones.

Can I cache multi-turn conversation history, not just the system prompt?

Yes. You can place the cache_control marker at the end of any content block — including previous messages in the conversation history. This is useful for long-running sessions where the earlier turns are stable within a call but change between sessions. Each placement requires a separate cache_control marker; Anthropic supports up to four simultaneous cache breakpoints per request. (docs.anthropic.com, 2025)

What is the minimum prefix size?

On Anthropic, it is model-dependent: typically 1,024 tokens for Haiku-tier models and higher for Opus-tier. On OpenAI, 1,024 tokens across all eligible models. If your stable prefix is shorter than the minimum, the cache write is silently skipped — you will not see an error, just zero cache activity in the usage fields. Check your prefix length if you are not seeing any cache_creation_input_tokens activity.

Where this fits in the series

This tutorial is episode 5 of The Hidden Cost of AI Coding. The earlier episodes established that resent context is the dominant cost driver in multi-step agents; this episode shows the primary remedy. The next step after caching is model tiering — routing lower-complexity tasks to Claude Haiku 4.5 instead of Sonnet 4.6 or Opus 4.8, which is the subject of episode 6 in the series. Browse all tutorials to follow the full series.