RAG vs MCP: Which Is Which, and When to Use Each

July 24, 2026 · Claude Certified Architect (CCA) Prep (part 6)

▶ Watch on YouTube & subscribe to The Stack Underflow

Somewhere in every “let’s add AI to the product” kickoff meeting, someone says “we’ll need RAG” and someone else says “we’ll need MCP” and both of them nod like they mean the same thing. They don’t. One fills in what the agent knows. The other grants what the agent can do. Confuse them and you’ll spend a sprint building a vector store to solve a problem that needed a tool call — or worse, wiring up a tool to solve a problem that needed a paragraph of retrieved text.

This is Q3 of 10 in the Domain 1 practice set — Agentic Architecture & Orchestration, 27% of the CCA-F exam, the heaviest of the five domains. It’s also the single most re-shareable idea in the set, because the mechanism generalizes to almost every agent design decision you’ll make.

The one-sentence version: RAG fills the context window with retrieved knowledge; MCP gives the model a standard port to call tools and take action. They solve different problems and usually work together, not instead of each other.

The question

In an agent, RAG and MCP do different jobs. Which is which?

• A) Both retrieve documents
• B) RAG retrieves knowledge; MCP exposes tools/actions
• C) MCP retrieves; RAG acts
• D) They’re synonyms

Pause here and pick before you scroll. If you can’t justify your answer in one sentence, you don’t know it yet — you’re guessing.

The answer

The correct option is B — RAG retrieves knowledge; MCP exposes tools/actions.

The cleanest way to see why is to stop thinking of RAG and MCP as competing technologies and start thinking of the context window as a budget with named line items. Every agent request has to fit a finite window, and each mechanism fills a different line:

CONTEXT WINDOW — BUDGET VIEW

┌─────────────────────────────────────────────────────┐
│ [ system prompt ]                                    │
│ [ tools/schemas  ]  ←── MCP: grants CAPABILITY        │
│ [ retrieved docs ]  ←── RAG: grants KNOWLEDGE         │
│ [ conversation history ]                              │
│ [ working memory / scratchpad ]                       │
└─────────────────────────────────────────────────────┘

RAG   → answers "what does the model KNOW right now?"
MCP   → answers "what can the model DO right now?"

RAG (Retrieval-Augmented Generation) is a knowledge mechanism. At request time it embeds the query, searches a vector store (or hybrid search index) for relevant chunks, and injects the retrieved text into the context window before generation. It changes what the model can talk about — grounding it in your docs, your support tickets, your codebase — without retraining anything. No action happens. Nothing changes outside the conversation. RAG is read-only by construction.

MCP (Model Context Protocol) is a capability mechanism. It’s a standard interface — the USB-C port analogy from earlier in the series — through which a model discovers and calls tools: hit an API, write a row, send an email, run a query, deploy a change. MCP doesn’t hand the model facts; it hands the model a menu of actions with typed schemas, and the model decides when to invoke one. Tool calls can read data too (a search_tickets tool, for instance) — but the defining trait of MCP is that it’s the model acting through an interface, not text silently appended to the window.

So the mechanism split is: RAG is a data pipeline that runs before the model speaks and pads its knowledge. MCP is a protocol the model actively invokes mid-reasoning to change or observe the world. One is passive enrichment; the other is agency.

Why the other options fail

• A) Both retrieve documents. This collapses the distinction that the whole question is testing. MCP can expose a document-search tool, but that’s one possible tool among many — MCP itself is a transport and interface standard for calling any capability, not a retrieval mechanism. Calling MCP “retrieval” mistakes one possible use case for the whole protocol.
• C) MCP retrieves; RAG acts. This is the mechanism exactly backwards. RAG has no ability to act — it has no tool-call loop, no side effects, no write path. It searches an index and pastes text into a prompt. MCP is the one with an execution surface. Swap the verbs and the sentence describes nothing real.
• D) They’re synonyms. They aren’t even in the same category of thing. RAG is a retrieval technique (embeddings, vector search, chunking, reranking). MCP is a protocol for tool discovery and invocation (JSON-RPC style messages, typed schemas, servers and clients). A technique and a protocol can compose in the same agent — most production systems use both — but “compose well together” is the opposite of “synonym.”

The concept behind it

Zoom out and this question is really testing whether you can classify any agent capability into one of two buckets: does it change what the model knows, or does it change what the model can do? That classification decides your entire design:

The trap the exam (and real architecture reviews) sets is presenting a vague requirement — “the agent needs access to our product data” — and expecting you to notice it’s ambiguous until you ask: is this read (RAG) or read-and-write-with-consequences (MCP, probably with a tool)? A support bot that quotes your refund policy needs RAG. A support bot that actually issues the refund needs MCP with a tool, and probably a human-in-the-loop gate given how destructive that action is — see designing safe destructive tools for that follow-on problem.

They also compose constantly. A common production pattern: MCP exposes a search_knowledge_base tool that, under the hood, runs a RAG pipeline — vector search plus reranking — and returns the top chunks as the tool result. From the model’s point of view it called a tool; from the system’s point of view, RAG did the retrieval work behind that tool’s interface. RAG can live inside an MCP tool. That’s not a contradiction of the split above — it’s the split working at two different layers: the outer layer is “the model invoked a capability” (MCP), the inner implementation of that capability happens to be a retrieval pipeline (RAG). If you found this framing useful, the deeper explainers on MCP in one diagram and RAG vs context engineering go further into each side independently.

The other place this distinction bites people is context budgeting. Because both mechanisms ultimately put tokens into the same finite window, teams that don’t separate “knowledge tokens” from “tool-schema tokens” in their mental model end up unable to diagnose why the window is full. If your agent’s context keeps overflowing, the first question is which line item is actually growing — retrieved documents, or an ever-expanding tool catalog. That diagnosis is exactly the subject of decomposing before you overflow the context window.

Not affiliated with, authorized, or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic. These are original practice questions for study, not real exam content — verify current exam details on Anthropic’s official pages.

FAQ

Do I need both RAG and MCP in the same agent? Most non-trivial production agents end up with both, because most real tasks require both knowing something and doing something. A customer-support agent might use RAG to ground its answer in your help docs and an MCP tool to actually update the customer’s account. They’re not mutually exclusive — they answer different halves of “what does this agent need to function?”

Is MCP a replacement for RAG? No. This is a common misreading of “RAG is dead” takes that actually mean “naive always-retrieve RAG is being replaced by smarter context engineering,” not “retrieval is obsolete.” MCP standardizes tool access; it has nothing to say about whether you should retrieve documents. See RAG vs context engineering for the fuller argument.

If an MCP tool does a document search, isn’t that just RAG wearing a different hat? Functionally, the retrieval math (embed, search, rerank) can be identical. The distinction that matters for architecture is the interface: RAG-as-pipeline runs automatically before generation and its output is silently appended to context; RAG-as-MCP-tool is something the model explicitly decides to invoke, with a visible tool call and a typed result. That difference matters for auditability, cost control, and letting the model choose not to search when it already knows the answer.

How much of the CCA-F exam is this kind of RAG-vs-MCP conceptual question? Domain 1 (Agentic Architecture & Orchestration) is roughly 27% of the exam per public exam-blueprint framing, and questions distinguishing retrieval from tool-use recur across that domain and into Domain 2 (Tool Design & Integration). As of mid-2026 the exam runs on Pearson VUE, costs $125, allows retakes, and is 60 questions in 120 minutes with a pass mark of 720/1000 — verify current numbers on Anthropic’s official pages before you register.

Where this fits in the CCA series

This is question 3 of 10 in the Domain 1 practice set. For the full five-domain map, start with the CCA exam guide. Two explainers go deeper on each half of this question: MCP explained on one diagram and RAG vs context engineering. Staying in Domain 1, see how the same “capability vs knowledge” thinking plays out in exposing 30 tools across 3 services, what happens when the agent ignores what a tool observes, and how to decompose before your context window overflows. Browse all tutorials for the rest of the series.