RAG vs Context Engineering: Why 'RAG Is Dead' and What Replaces It

July 24, 2026 · Claude Certified Architect (CCA) Prep (part 3)

▶ Watch on YouTube & subscribe to The Stack Underflow

For two years, the answer to “make the AI smarter” was the same: bolt on a vector database, embed your documents, retrieve the top-k chunks, stuff them into the prompt. That was RAG, and it worked well enough that “add RAG” became a reflex instead of a decision. Then, in 2026, even the researchers who popularized retrieval-augmented generation started saying the real skill was something bigger. RAG did not die. It got demoted — from the whole strategy to one line-item in a bigger budget. That budget is your context window, and spending it well is what the industry now calls context engineering.

If you are prepping for the Claude Certified Architect exam, this reframe is not trivia. It is the spine of Domain 5 (Context Management & Reliability, roughly 15% of the exam) and it runs directly into Domain 4 (Prompt Engineering & Structured Output, roughly 20%). Get the mental model right here and a large share of the exam’s scenario questions become mechanical.

The one-sentence version: The context window is a budget, not a memory — RAG fills what the model knows, tools (via MCP) grant what it can do, and context engineering is the discipline of spending the window on what actually earns the answer.

The old answer, and why it stopped being enough

RAG (retrieval-augmented generation) solved a real problem: language models are frozen at training time and have no access to your private, current, or proprietary data. The fix was elegant — embed your documents, store the vectors, retrieve the closest matches to a query, and inject them into the prompt so the model can ground its answer in something it never trained on.

That pattern is still correct. What was wrong was treating it as the entire answer to “make the AI smarter.” Teams that only ever reached for RAG ran into three recurring failures:

• Irrelevant retrieval crowds out everything else. A retriever tuned for recall pulls in five loosely-related chunks, and now half the context window is noise competing with the system prompt and the actual task.
• RAG has nothing to say about tools. If the agent needs to do something — call an API, run a query, send an email — retrieval alone cannot grant that capability. That is a different pillar entirely (this is where MCP enters).
• RAG has nothing to say about memory across turns or sessions. What happened three messages ago, or in yesterday’s conversation, is not a retrieval problem — it is a state problem.

Once you name these three gaps, the fix is obvious: RAG needed to be re-framed as one tool in a larger system, not the system itself.

The reframe: RAG shrinks into one box

Picture a large frame labeled CONTEXT. Retrieval — RAG — used to be drawn as the whole frame. Now it is one box inside it, sitting alongside instructions, memory, and tools.

BEFORE (2024-era mental model)         AFTER (2026 mental model)
┌─────────────────────────┐           ┌───────────────────────────────────┐
│                         │           │              CONTEXT               │
│           RAG           │    ──▶    │  ┌───────────┐ ┌─────┐ ┌────────┐  │
│    (the whole answer)   │           │  │INSTRUCTIONS│ │ RAG │ │  TOOLS │  │
│                         │           │  └───────────┘ └─────┘ └────────┘  │
└─────────────────────────┘           │            ┌──────────┐            │
                                       │            │  MEMORY  │            │
                                       │            └──────────┘            │
                                       └───────────────────────────────────┘

RAG did not get worse at what it does. The scope of the problem got bigger, and RAG is now correctly sized as one answer to one part of it — “what facts does the model lack right now?” — instead of the answer to everything.

The context window is a budget, not a memory

Here is the reframe that the exam actually tests: stop thinking of the context window as a growing scratchpad that remembers things for you. Think of it as a fixed-width budget that every piece of information has to compete for, token by token.

CONTEXT WINDOW (finite, N tokens)
┌──────────────┬─────────┬──────────────────┬───────────┬──────────┐
│ SYSTEM PROMPT │  TOOLS  │  RETRIEVED DOCS  │  HISTORY  │  MEMORY  │
│  (who it is)  │ (what it│   (RAG output —  │ (this run,│(long-term│
│               │ can DO) │  knowledge it     │ short-term)│  store)  │
│               │         │  lacks)           │           │          │
└──────────────┴─────────┴──────────────────┴───────────┴──────────┘
      $ / tokens ─────────────────────────────────────────▶

Every segment you widen is a segment you narrow somewhere else. Dump in ten retrieved chunks “just in case,” and you have less room for conversation history, fewer tool schemas you can afford to expose, or a truncated system prompt. There is no free lunch inside a fixed window — this is precisely why “add more context” is not a strategy, and why the exam frames curation, not accumulation, as the skill being tested.

The four pillars of context engineering

Context engineering names four distinct levers. The exam expects you to recognize which lever a scenario is describing, because each has a different failure mode and a different fix.

A few distinctions worth being precise about, because the exam likes to test the boundary:

• Instructions vs. memory. Instructions are stable — they describe identity and behavior and rarely change mid-conversation. Memory is dynamic — it is what was said or decided, and it changes every turn.
• Retrieval vs. tools. Retrieval (RAG) fills in knowledge — facts the model was never trained on. Tools grant capability — the ability to take an action. A tutorial recommendation system needs retrieval; an agent that books a meeting needs a tool. Many real systems need both, and mixing them up is a common wrong-answer trap: retrieving a document is not the same operation as calling an API, even when both happen inside the same turn.
• Short-term vs. long-term memory. Short-term memory lives inside the current context window — it’s the conversation history segment in the diagram above, and it evaporates when the session ends. Long-term memory lives in an external store (a database, a vector index, a file) and gets pulled back in deliberately, the same way retrieved documents do. If a scenario describes something that must survive across sessions, the answer is a durable store, not “just keep it in context.”

Overflow and compaction: what happens when you blow the budget

A fixed-width budget can be overrun. When the sum of instructions, tools, retrieved docs, history, and memory exceeds the window, something has to give — and if you don’t decide what, the platform decides for you, usually by silently truncating from the middle or the start.

Budget healthy:     [SYSTEM][TOOLS][DOCS][HISTORY][MEMORY]           ▮▮▮▮▮▮▮▮▯▯  quality: stable
Budget overflowing: [SYSTEM][TOOLS][DOCS][HISTORY..............][MEM]▮▮▮▮▯▯▯▯▯▯  quality: dropping
                                          └── uncontrolled growth ──┘
Compaction applied:  [SYSTEM][TOOLS][DOCS][hist-summary][MEMORY]     ▮▮▮▮▮▮▮▯▯▯  quality: recovers
                                          └─ pruned / summarized ──┘

Compaction is the deliberate practice of summarizing, pruning, or rotating out low-value tokens before the window forces the issue — condensing older conversation turns into a summary, dropping retrieved chunks that scored low, or archiving stale tool results. Note the emphasis on deliberate: this is exactly the domain-5 territory covered in depth in Your Agent Blew the Context Window: Decompose Before You Overflow and The Task Overflowed the Window: Chunking and Decomposition. The point to hold onto here is narrower: overflow is not a crash, it’s a quality curve, and compaction is what keeps you on the healthy side of it.

The rule: more context is not better context

This is the line the exam wants you to internalize: signal per token matters more than total tokens. Two systems can have identical context-window sizes and wildly different output quality, because one is spending its budget on relevant, load-bearing information and the other is spending it on noise.

MORE context   →  ████████████████████░░  quality: flat or declining (drowned in irrelevant tokens)
RIGHT context  →  ████████░░░░░░░░░░░░░░  quality: high (every token earns its place)

This is why “just retrieve more chunks” or “just paste the whole document” is almost never the fix to a quality problem, and why the exam consistently rewards answers that involve curating context (better retrieval ranking, scoped tools, summarized history) over answers that involve adding more of it. If you’ve seen the phrase “lost in the middle” — where a model under-weights information buried in a long context — that’s the concrete failure mode behind this rule, covered in It Forgot Your First Instruction: Lost-in-the-Middle and Context Rot.

Where RAG and MCP actually sit

Two follow-on questions come up constantly once this reframe clicks, and both have exam-relevant answers:

Is RAG obsolete? No. RAG is still the correct answer whenever the model needs facts it wasn’t trained on — internal docs, current pricing, a customer’s account history. What changed is that RAG is now one pillar feeding one segment of the budget, not a synonym for “the system’s intelligence.” For when retrieval is the right tool versus when it isn’t, see RAG vs MCP: Which Is Which, and When to Use Each.

Where does MCP fit? MCP (Model Context Protocol) is the standardized way of exposing the tools pillar — it’s how an agent’s capabilities get described and invoked consistently, the same way RAG is how the retrieval pillar gets implemented. If RAG answers “what does it know,” MCP answers “what can it do.” The full mental model for that pillar is in MCP Explained on One Diagram: The USB-C Port for AI Tools.

Common misconceptions

• “RAG is dead” means retrieval doesn’t matter anymore. It does not mean that. It means retrieval is no longer the entire strategy — it’s a scoped answer to a scoped question (missing knowledge), and it now has to compete for budget with instructions, tools, and memory rather than being assumed to fill the whole window.
• A bigger context window solves context problems. A larger window raises the ceiling on the budget; it does not change the rule that signal-per-token beats total-tokens. Teams that scale up window size without improving curation typically see quality plateau or regress, because low-value tokens simply scale up alongside the useful ones.
• Memory just means “keep everything in the conversation.” Conflating short-term (in-window) memory with long-term (external store) memory is one of the most common wrong answers on scenario questions. If information needs to persist beyond the current window, it belongs in a store you deliberately retrieve from — not in an ever-growing history segment.
• Tools and retrieval are interchangeable. They answer different questions — knowledge versus capability — and a scenario that needs an action taken (send, delete, book, update) is a tools problem even if it superficially resembles a retrieval problem (“find and update the record”).

Frequently asked questions

What is context engineering, in one sentence? Context engineering is the discipline of deciding what goes into a model’s finite context window — instructions, retrieved knowledge, memory, and tools — and what gets left out, so that every token spent competes for and earns its place.

Did RAG actually get replaced by something else? No single technique replaced RAG. It was re-scoped: RAG remains the standard pattern for grounding a model in facts it wasn’t trained on, but it’s now one of four pillars (instructions, retrieval, memory, tools) rather than the whole architecture. Context engineering is the umbrella term for coordinating all four.

How is context engineering different from prompt engineering? Prompt engineering historically focused on wording — how you phrase an instruction to get a better single-turn response. Context engineering is broader: it’s about what information populates the window across an entire interaction (or an entire agent run), including retrieval, memory, and tool availability, not just phrasing. On the exam, prompt engineering topics sit inside the broader Domain 4 umbrella, while the context-budget mechanics live in Domain 5.

What happens if I just make the context window bigger instead of curating it? You raise the ceiling, but you don’t change the underlying rule. A bigger window with the same low signal-per-token ratio typically shows flat or declining quality, because irrelevant retrieved chunks, stale history, and unused tool schemas all scale up along with the useful content. Curation — precise retrieval, scoped tools, summarized history — is what actually improves output quality; window size alone is not.

Is this on the actual Claude Certified Architect exam? The context-budget model maps directly to Domain 5 (Context Management & Reliability) and Domain 4 (Prompt Engineering & Structured Output). As of mid-2026 the exam is delivered via Pearson VUE, costs $125 per attempt with retakes available, runs 60 questions in 120 minutes, and requires a scaled score of 720 out of 1000 to pass — always verify these logistics on Anthropic’s official pages, since exam details can change. For the full five-domain breakdown, see the Claude Certified Architect (CCA) Exam Guide.

Not affiliated with, authorized, or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic. These are original practice questions for study, not real exam content — verify current exam details on Anthropic’s official pages.

Where this fits in the series

This tutorial is the third episode in the CCA prep intro arc, following the Claude Certified Architect (CCA) Exam Guide (all five domains on one diagram) and MCP Explained on One Diagram (the tools pillar in depth). Together, these three pieces give you the vocabulary the rest of the series builds on: the exam’s structure, what MCP grants an agent, and how the context window budgets everything an agent knows and does. Don’t memorize the answer — understand the architecture. Browse all tutorials to keep going through the series.