The Task Overflowed the Window: Chunking and Decomposition

July 24, 2026 · Claude Certified Architect (CCA) Prep (part 45)

▶ Watch on YouTube & subscribe to The Stack Underflow

Your agent is twenty steps into a task that should take fifty. It has read six files, called eight tools, and is holding a full transcript of every intermediate result. Then the next tool call fails — not because the logic is wrong, but because the request no longer fits. The context window, a fixed budget of tokens, is full. This is not a rare edge case in agentic work; it is the default failure mode of any long-running loop that never manages its own memory. The question is what you reach for first.

The one-sentence version: When a multi-step task overflows the context window mid-run, the first lever is compaction — summarizing the accumulated history into a scratchpad so the run keeps its thread without keeping every token.

The question

A multi-step task overflows the context window mid-run. First lever?

• A) A bigger model
• B) Summarize/compact the history into a scratchpad
• C) Delete the system prompt
• D) Restart

Pause here and pick an answer before you keep reading.

The answer

The correct option is B) Summarize/compact the history into a scratchpad.

The context window is not a suggestion, it is a hard budget. Every message, tool call, and tool result an agent accumulates during a run consumes part of that budget, and unlike a human’s memory, nothing falls away on its own. Left unmanaged, a long-running agent loop grows its history monotonically until it hits the ceiling — which is exactly the scenario in the question.

The mechanism that recovers room without losing the thread is compaction: at some point in the run — either proactively, on a schedule, or reactively, when usage crosses a threshold — the agent (or an orchestration layer around it) takes the accumulated conversation history and summarizes it into a condensed form. That summary, often called a scratchpad, replaces the verbose transcript. The system prompt, the tools, and the current task stay intact; only the bulky, already-processed history gets compressed.

CONTEXT WINDOW — a fixed token budget

BEFORE (mid-run, overflowing)
┌───────────────────────────────────────────────────────────┐
│ SYSTEM │ TOOLS │ ░░░░░░░░ HISTORY (grows every turn) ░░░░░░░│▓▓▓▓
└───────────────────────────────────────────────────────────┘
                                                          ^ next call can't fit

FIRST LEVER — compact the history into a scratchpad
┌───────────────────────────────────────────────────────────┐
│ SYSTEM │ TOOLS │ SUMMARY (compacted) │ TASK │  free room    │
└───────────────────────────────────────────────────────────┘
          HISTORY ──[compact/summarize]──▶ SUMMARY
          room recovered · thread preserved

Notice what compaction is not: it is not deleting the history and hoping nothing important was lost. A good compaction step preserves decisions made, facts established, and open sub-goals — the things later steps depend on — while dropping the raw tool-call noise that got the agent there. The run continues with a smaller, denser footprint instead of a smaller, amnesiac one.

Why the other options fail

• A) A bigger model. A larger context window buys you a higher ceiling, not a fix for the underlying growth pattern. An agent that accumulates history without bound will eventually overflow any window — it just takes longer. This also does nothing for the run that is overflowing right now; you cannot swap models mid-request to retroactively make room, and larger windows carry real cost and latency tradeoffs of their own.
• C) Delete the system prompt. The system prompt is usually a small, fixed cost compared to a growing history — deleting it frees little room while removing the instructions, tool contracts, and guardrails the agent needs to behave correctly for the rest of the run. You would trade a token-budget problem for a reliability problem, and the token savings wouldn’t even solve the original overflow.
• D) Restart. Restarting clears the window, but it also clears every step of progress the run had made. For a genuinely multi-step task, that is not a fix — it’s a reset button that guarantees you pay the same cost again, with no assurance the same overflow won’t recur at the same step.

The concept behind it

Context-window overflow in agentic systems is really two problems wearing one name, and the fix depends on which one you’re facing.

The first is runaway accumulation within a single run — the scenario in this question. A long agent loop keeps appending tool results and turns to its history without ever pruning. The fix is compaction: periodically summarizing the older parts of the history into a scratchpad, so the working context stays bounded while the thread of the task survives. This is a reactive discipline — it happens during the run, in response to usage pressure.

The second problem is a task that was too big to fit in one context from the start. No amount of compaction saves you if step one requires ingesting more source material than the window can hold. The fix there is decomposition (also called chunking): breaking the task into smaller subtasks, each scoped to its own bounded context, often handed to separate agent calls or subagents that only see what they need for their slice of the work. This is a proactive discipline — it happens at design time, before the run starts, so overflow never becomes a mid-run emergency in the first place.

The two techniques compose. A well-designed agentic system decomposes a large task into chunks small enough that any single chunk’s context stays comfortably under budget, and within each chunk’s run, it still compacts history as the loop goes long — because even a well-scoped subtask can run longer than expected. Treating the context window as a fixed budget, the way you’d treat memory or disk space in traditional systems engineering, is the mental model that makes both techniques obvious rather than exotic. It’s the same discipline covered from a different angle in 80% History, 20% Task: Budgeting the Context Window — overflow is what happens when that budget goes unmanaged for too many turns.

One detail worth internalizing: a compacted scratchpad that lives only in the model’s context is still fragile. If the orchestration layer later performs a hard reset of the conversation — a new session, a crashed process, a context-window rollover — an in-context scratchpad disappears with it. Durable agent systems persist that scratchpad to external state (a database row, a file, a memory store) rather than trusting it to survive purely inside the window. That’s a distinct but closely related failure mode, covered in Your Scratchpad Vanished on Compaction: Persisting Agent State.

Compaction also interacts with a second well-known context failure: even when everything technically fits, position matters. Content buried in the middle of a long, uncompacted history is exactly the material an attention mechanism is least likely to weight correctly — see It Forgot Your First Instruction: Lost-in-the-Middle and Context Rot for the mechanism. Compacting the history doesn’t just save tokens; it also removes stale middle-of-context material that was actively degrading recall, which is part of why compaction tends to improve answer quality on long runs, not just reduce cost.

FAQ

Is compaction the same thing as decomposition? No. Compaction acts on history that already accumulated during a run — it summarizes the past to make room for the future. Decomposition acts at design time, before the run starts — it splits an oversized task into smaller pieces so each piece’s context never needs to grow that large. Both defend the same budget, from opposite directions.

Does upgrading to a model with a larger context window solve this permanently? It raises the ceiling, not the discipline. Any agent loop that appends history without pruning will eventually overflow a bigger window too — it just takes more turns to get there, and you pay for those extra tokens along the way. Compaction and decomposition are the fixes that scale independent of window size.

Who decides when to trigger compaction — the model or the orchestration layer? Both patterns exist in production systems. Some designs compact reactively, when token usage crosses a threshold (say, 80% of the window). Others compact proactively on a fixed schedule (every N turns), which trades a small amount of unnecessary summarization for predictability. Either way, the trigger logic lives in the orchestration layer around the model, not inside the model’s own reasoning.

What should a compaction step actually preserve? Decisions already made, facts already established, and any open sub-goals the rest of the run still depends on. What it should discard is the raw mechanics of how the agent got there — full tool call arguments, verbose intermediate outputs, exploratory dead ends. A good summary reads like a status report a human hands off mid-shift, not a transcript.

Where this fits in the CCA series

This is question 2 of 10 in the Domain 5 (Context Management & Reliability, 15% of the exam) practice set. As of mid-2026 the CCA is administered via Pearson VUE, costs $125 with retakes available, and runs 60 questions in 120 minutes with a passing scaled score of 720/1000 across five domains — verify current details on Anthropic’s official pages before you register.

For the theory this question sits inside, see RAG vs Context Engineering: Why ‘RAG Is Dead’ and What Replaces It, and for the full map of what’s tested, start with the keystone: Claude Certified Architect (CCA) Exam Guide: All Five Domains on One Diagram. Browse all tutorials for the rest of the series.

Not affiliated with, authorized, or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic. These are original practice questions for study, not real exam content — verify current exam details on Anthropic’s official pages.