Why Is Your Cache Never Hitting? Prompt Cache Prefix Rules

July 24, 2026 · Claude Certified Architect (CCA) Prep (part 47)

▶ Watch on YouTube & subscribe to The Stack Underflow

You did everything the docs told you to do. You added cache_control to the stable block at the top of your prompt, deployed it, and watched your dashboard for the cost drop. It never came. cache_creation shows tokens being written every single call, but cache_read sits at a flat, permanent zero. No errors. No warnings. The API just quietly bills you full price, call after call, as if the cache isn’t there at all. That silence is the trap — caching doesn’t fail loudly, it fails invisibly, and the fix is one number you probably haven’t checked.

The one-sentence version: If your cacheable prefix is shorter than the model’s minimum cacheable length, the API accepts the cache_control marker but never actually caches anything — no error, just a permanent cache_read of zero.

The question

You added caching to your application, but after several calls, cache_read stays at 0 — no hits are registering, even though you marked a prefix with cache_control. What is the most likely cause?

• A) Caching is broken
• B) The cacheable prefix is below the model’s minimum (e.g. Haiku 4.5 = 4096 tokens) — enlarge it
• C) Wrong model
• D) Temperature too high

Pause here and pick an answer before you scroll — this is Q4 of 10 in the Domain 5 (Context Management & Reliability) practice set for the Claude Certified Architect exam.

The answer

The correct answer is B. The cacheable prefix you marked is smaller than the model’s minimum cacheable-prefix length, so the API silently skips caching it. There’s no error because, from the API’s point of view, nothing went wrong — the request just didn’t contain enough content in that block to be worth caching, and it says so only by omission.

Here’s the mechanism. Every model that supports prompt caching enforces a floor — a minimum token count a marked prefix must clear before the platform will bother writing it to the cache. Content below that floor is processed normally on every call: no cache_creation, no cache_read, no discount. The floor exists because caching has overhead (writing the cache entry, matching it on the next call), and below a certain size that overhead isn’t worth paying — so the platform just no-ops instead of erroring.

CACHEABLE PREFIX LENGTH vs THE CACHE FLOOR

 tokens
   │
   │              ┌────────────────────────┐
   │              │   YOUR SYSTEM PROMPT +  │  ← well above the floor
   │              │   TOOL DEFS + POLICY    │     → cache_creation writes it
   │              │   (e.g. 6,000 tokens)   │     → next call: cache_read hits
   │              └────────────────────────┘
   │
   ┼ ─ ─ ─ ─ ─ ─ ─ ─ CACHE FLOOR ─ ─ ─ ─ ─ ─ ─ ─   (model-specific minimum)
   │                                                 e.g. Haiku 4.5 ≈ 4096
   │   ┌──────────┐                                 e.g. Sonnet 4.x ≈ 1024
   │   │ your 300-│  ← below the floor
   │   │ token     │     → cache_control is accepted
   │   │ prefix    │     → but NOTHING is cached
   │   └──────────┘     → cache_read stays 0, forever
   │
   └──────────────────────────────────────────────► requests over time

Fix: pad the marked block with real, stable content (full tool schemas,
full policy text, few-shot examples) until it clears the floor line.

The fix is mechanical once you see it: enlarge the cacheable prefix. That doesn’t mean pad it with junk — it means pull genuinely stable content into the cached block that you were previously sending fresh every time: full tool definitions instead of trimmed ones, the complete policy document instead of a summary, a block of few-shot examples. Once the marked prefix clears the model’s minimum, cache_creation fires on the first call and cache_read starts showing hits on every call after.

Exact floor numbers are model-specific and change as new models ship — treat any number you see (including the ones above) as something to verify against Anthropic’s current prompt-caching documentation before you rely on it in production, but the shape of the rule — a hard minimum below which caching silently no-ops — is stable across the Claude model family.

Why the other options fail

• A) “Caching is broken.” This is the instinctive answer because the symptom looks like a bug: you configured something and it did nothing. But “broken” implies an error, a rejected request, or an exception — none of which happen here. The API validated your request and executed it correctly; it simply didn’t meet the size threshold to cache. Assuming breakage sends you down the wrong debugging path (retrying, re-deploying, filing a support ticket) instead of the right one (measuring your prefix length).
• C) “Wrong model.” Model choice matters indirectly — different models have different floors, and Haiku’s floor is notably higher than Sonnet’s — but simply using “the wrong model” isn’t the root cause of a zero-hit cache. Even the “right” model will show the same zero if the marked prefix is too short for that model’s floor. The lever is prefix size, not model swap.
• D) “Temperature too high.” Temperature controls sampling randomness in the output — it has no relationship to how the API decides what to cache. This option is a classic distractor: it’s a real, plausible-sounding tuning knob that happens to be completely irrelevant to the mechanism being tested. If you find yourself reaching for a sampling parameter to explain a caching problem, you’ve mixed up two unrelated layers of the API.

The concept behind it

Prompt caching exists to solve one specific waste: sending the same large, unchanging block of tokens — a system prompt, a tool catalog, a style guide, a knowledge base excerpt — on every single call, and paying full input-token price to re-process it every time. You mark the end of that stable block with cache_control, and on subsequent calls with an identical prefix, the model reads from cache instead of reprocessing, at a fraction of the cost and with lower latency. It is one of the highest-leverage cost optimizations available in context engineering, precisely because it’s nearly free to implement — a single flag — and pays off on every repeated call.

But “nearly free to implement” hides a gotcha: prompt caching is not automatic just because you added the flag. It’s gated by a size threshold, and that threshold is the single most common reason a real caching rollout underperforms its cost-savings projection. Three practical rules follow from the mechanism:

The deeper lesson for the exam — and for production systems — is that reliability failures in LLM applications are frequently silent. Nothing throws, nothing logs an error, the request succeeds — and the system just quietly performs worse than you designed it to. Domain 5 tests you on exactly this pattern repeatedly: a context window that silently loses signal in the middle, a scratchpad that silently vanishes on compaction, a subagent failure that gets silently swallowed. Prompt caching’s floor is the same failure shape applied to cost and latency instead of correctness. The habit that catches all of them is the same: instrument the metric that would reveal the silent failure (cache_read, in this case) and check it, rather than assuming a feature works because you configured it.

FAQ

How do I know if my cache is actually hitting? Check the cache_creation_input_tokens and cache_read_input_tokens fields in the API response usage block. The first call after a change should show cache_creation tokens written; every subsequent identical-prefix call should show cache_read tokens instead of full input_tokens. If cache_read stays at zero across many calls with an unchanged prefix, the prefix likely isn’t clearing the model’s minimum.

Does a bigger cached prefix always mean bigger savings? Only up to a point, and only if the block is genuinely stable across calls. Padding a prefix with content that changes even slightly between calls (a timestamp, a per-user field) breaks the exact-match requirement and forces a fresh write every time — worse than not caching, since you pay the cache-write overhead with none of the read benefit. Cache what is truly invariant; keep anything variable after the cache boundary.

Is the cache floor the same across all Claude models? No — it varies by model family and is expected to change as new models release, so treat any specific number as a snapshot to verify, not a constant to hardcode. The pattern to remember for the exam is the existence of a minimum, not the specific figure.

Why doesn’t the API just warn me if my prefix is too short? Because “too short to cache” isn’t an error state from the API’s perspective — it’s a valid request that simply doesn’t meet an optimization threshold. The API’s job is to answer the request correctly, not to audit your cost-efficiency choices. That’s exactly why you need to watch the usage metrics yourself rather than trusting the absence of errors as a health signal.

Where this fits in the CCA series

This question builds directly on Stop Re-Sending the Same Big Block: Prompt Caching for Context, which covers why and what to cache — this one covers the specific failure mode of caching that looks configured but never actually fires. It sits in the same reliability cluster as It Forgot Your First Instruction: Lost-in-the-Middle and Context Rot and 80% History, 20% Task: Budgeting the Context Window — all three are about the context window silently failing to deliver value even when the request technically succeeds. For the caching mechanism in the context of the broader shift from retrieval to engineered context, see RAG vs Context Engineering.

This is question 4 of 10 in the Domain 5 practice set (15% of the exam). For the full five-domain map and how this question fits the bigger picture, start with the Claude Certified Architect (CCA) Exam Guide. Browse all tutorials for the rest of the series.

Not affiliated with, authorized, or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic. These are original practice questions for study, not real exam content — verify current exam details on Anthropic’s official pages.