It Forgot Your First Instruction: Lost-in-the-Middle and Context Rot

July 24, 2026 · Claude Certified Architect (CCA) Prep (part 44)

▶ Watch on YouTube & subscribe to The Stack Underflow

You set a rule at the very start of an agent session — “always confirm before deleting a file,” “respond only in JSON,” “never quote a price without checking the current rate card.” Forty turns later, the agent deletes the file without asking, replies in prose, or quotes last quarter’s price. Nobody touched the system prompt. Nothing crashed. The model just stopped acting like it remembered. That’s not a bug in the traditional sense — it’s a predictable property of how transformer attention behaves over long context, and it’s exactly the kind of scenario Domain 5 of the Claude Certified Architect exam is built to test.

The one-sentence version: Attention over long context isn’t uniform — it’s strongest at the start and end of the window and weakest in the middle, so instructions planted early and never reinforced quietly lose their pull; the fix is to keep anything critical pinned at the edges, not buried in the scroll.

The question

A long agent run starts ignoring instructions you gave early on. What’s happening, and the fix?

• A) The model is broken
• B) “Lost in the middle” — attention favors the start and end; keep critical instructions at the edges
• C) Raise temperature
• D) A bigger model

Pause here and pick an answer before you scroll — the point of these practice questions is reasoning through the mechanism, not pattern-matching the letter.

The answer

The correct answer is B. “Lost in the middle” is a well-documented behavior: as a context window fills up, a model’s effective attention to any given token is not flat across the whole sequence. It’s high for tokens near the beginning (recency-independent, primacy-like) and high again for tokens near the end (freshest, closest to the generation point), but it sags in the middle. An instruction planted at turn 1 and never repeated is, forty turns later, sitting in that low-attention middle zone — technically still in the context window, but functionally quieter than content the model just saw.

ATTENTION WEIGHT ACROSS A LONG CONTEXT WINDOW

  high │█▇                                          ▅██
       │ █▇                                        ▅█
       │  █▆                                      ▅█
  weight│   █▅▃                                 ▂▄█
       │      ▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂
  low  │              ← the middle: signal decays →
       └──────────────────────────────────────────────
        start of context      ...turns...      end of context
        (system prompt,                        (most recent
         early rules)                            turns, live task)

  Rule planted here ──┐                    Rule re-planted here
  at turn 1            ▼                    or moved to system
  "confirm before      ✗ fades as the      prompt / recent turn
   deleting"             transcript grows    ✓ stays strong

The fix follows directly from the mechanism: don’t rely on a single mention buried early in a long transcript. Put durable, critical rules in the system prompt (which many implementations treat with structurally higher priority, and which sits at a fixed, privileged position rather than drifting into the “middle” as turns accumulate), and reinforce anything truly load-bearing in recent turns — a short recap, a tool-call guardrail restated near the point of use, or a compaction step (see below) that re-surfaces the rule instead of letting it erode. The mechanism is about position in the window, not about the model “forgetting” in a human sense — so the fix is also about position, not about brute-forcing more capability at the problem.

Why the other options fail

• A) “The model is broken.” This treats a well-characterized attention property as a malfunction. It isn’t a bug report — it’s the expected shape of attention over long sequences, and it happens across models and providers, not just Claude. Filing it as “broken” means you’ll never look for the actual lever (position), and the same failure will recur on the next long run.
• C) “Raise temperature.” Temperature controls sampling randomness at the output layer — how much the model diversifies its token choices. It has nothing to do with which parts of the input context the model attends to strongly. Raising it doesn’t move a buried instruction closer to the edges of the window; if anything, it adds noise to an already-degraded signal.
• D) “A bigger model.” A larger model may have a longer supported context window and somewhat flatter attention decay, but lost-in-the-middle is a documented property across model scales, including the largest ones. Swapping models is an expensive way to marginally soften a curve you can fix directly and cheaply by repositioning the instruction. It also doesn’t scale — the next long enough run reproduces the exact same failure on the bigger model.

The concept behind it

Lost-in-the-middle is one instance of a broader phenomenon sometimes called context rot: the observation that a model’s effective reliability on any given piece of context degrades as the surrounding context grows, even when the total content still technically fits inside the window. “Fits in the window” and “is being reliably used” are two different claims, and the exam (and real production systems) will keep testing whether you understand the gap between them.

A few things worth internalizing beyond this one question:

Three practical patterns follow from this table, and each has its own exam-relevant deep dive:

1. Pin, don’t bury. Anything that must hold for the entire session — output format, safety constraints, non-negotiable business rules — belongs in the system prompt or gets mechanically re-injected, not typed once early and trusted to persist.
2. Compact instead of letting the transcript grow unbounded. When a long-running agent summarizes its own history into a scratchpad, that’s also an opportunity to re-surface the rules that matter, moving them back toward the “recent” end of the window instead of letting them rot in the discarded middle. See The Task Overflowed the Window: Chunking and Decomposition for the mechanics of when and how to compact.
3. Budget the window like a resource, not a bottomless log. If most of your tokens are old conversation history, you’re both paying for it and diluting the model’s attention on what currently matters. See 80% History, 20% Task: Budgeting the Context Window for how to rebalance that ratio.

This also connects to cost engineering: content that’s stable and needs to be attended to reliably — like a system prompt or a policy block — is exactly the kind of content worth caching rather than re-sending raw every call. Stop Re-Sending the Same Big Block: Prompt Caching for Context covers that mechanism, and Why Is Your Cache Never Hitting? Prompt Cache Prefix Rules covers the prefix rules that make caching actually work.

Reliability and context management overlap constantly in Domain 5: a rule lost in the middle produces the same symptom as a subagent error that gets swallowed instead of surfaced — confident output that quietly diverges from what should have happened. If that failure mode interests you, see It Failed and Answered Confidently Anyway: Error Propagation in Agents.

FAQ

Is lost-in-the-middle a Claude-specific problem, or does it affect all LLMs? It’s a general property of transformer attention over long sequences, observed across providers and architectures, not something unique to Claude. Different models and context lengths shift exactly where the “sag” is worst and how steep it is, but the shape — strong at the edges, weaker in the middle — shows up broadly. Treat it as a property of the technology class, not a vendor defect.

Does a longer context window fix lost-in-the-middle? No, and this is the trap in option D above. A longer window lets you fit more content, but it doesn’t guarantee uniform attention across that content — it can actually widen the low-attention middle region in absolute token terms. A bigger window is a capacity increase, not a reliability guarantee; the position of your critical instructions still matters.

How is lost-in-the-middle different from the context window simply overflowing? Overflow is a hard limit — the content literally doesn’t fit and something gets truncated or rejected. Lost-in-the-middle happens well before that: everything technically fits, but the model’s effective use of tokens in the middle is weaker than tokens at the edges. Overflow is a capacity problem you fix with decomposition or compaction; lost-in-the-middle is an attention problem you fix with positioning. They compound in long-running agents, which is why Domain 5 tests both.

What’s the simplest fix I can apply today without re-architecting anything? Move anything non-negotiable into the system prompt, and if you’re managing a long-running conversation or agent loop, periodically re-state critical constraints in a recent turn rather than assuming a turn-1 instruction still has full weight forty turns later. It’s a cheap habit that directly targets the mechanism instead of working around it.

Where this fits in the CCA series

This is Q1 of the Domain 5 (Context Management & Reliability, 15% of the exam) practice set. Start with the Claude Certified Architect (CCA) Exam Guide for how all five domains fit together, and if the caching or budgeting angles above are new to you, RAG vs Context Engineering is a useful primer on why context management became its own discipline in the first place. From here, the rest of the D5 set builds directly on this mechanism: The Task Overflowed the Window: Chunking and Decomposition, Stop Re-Sending the Same Big Block: Prompt Caching for Context, and 80% History, 20% Task: Budgeting the Context Window.

Not affiliated with, authorized, or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic. These are original practice questions for study, not real exam content — verify current exam details on Anthropic’s official pages.

As of mid-2026, the CCA-F exam runs on Pearson VUE, costs $125, and allows retakes — the format is 60 questions in 120 minutes with a passing scaled score of 720/1000 across five domains. Confirm these logistics on Anthropic’s official certification pages before you register, since exam details change.

Browse all tutorials for the rest of the series.