Your Tool Returns 5,000 Rows: Paginating and Shaping Tool Responses

July 24, 2026 · Claude Certified Architect (CCA) Prep (part 21)

▶ Watch on YouTube & subscribe to The Stack Underflow

A query tool goes out to the database, does its job perfectly, and comes back with 5,000 rows of JSON. The tool call succeeded. The database is fine. And the agent run just got noticeably worse — slower, more expensive, and more likely to lose the thread of what it was doing three tool calls ago. Nothing crashed. That’s what makes this failure mode sneaky: it degrades quietly instead of erroring loudly, and a lot of teams ship it to production before anyone notices.

The one-sentence version: Tool output is not free — every row you return is a line-item charged against the context window, so the fix is to return a summary, a page, or a handle, not the raw dump.

The question

A query tool returns 5,000 rows and blows the context window. What’s the best fix?

• A) Bigger context window
• B) Return a summary, a page, or a handle/ID — not the raw dump
• C) Truncate randomly
• D) Call it twice

Pause here and pick an answer before you keep scrolling.

The answer

The correct choice is B — return a summary, a page, or a handle/ID, not the raw dump.

Here’s the mechanism. A tool result doesn’t get treated specially by the model just because it came from a database instead of a person typing a message. It goes into the exact same context window as everything else — the system prompt, the conversation history, and every other tool result in the run. If a tool hands back 5,000 rows of raw JSON, that JSON is now competing for space with the instructions that tell the model how to behave and the reasoning it needs to do its job.

CONTEXT WINDOW — a fixed budget, not a suggestion

┌────────────────────────────────────────────────────────┐
│ system prompt                                    [ 4%]  │
│ conversation history                              [18%]  │
│ ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░  [78%]  │ ← 5,000 raw rows
│ ← flooded. Little room left to reason about them.        │
└────────────────────────────────────────────────────────┘
                        │
                        │ instead, the tool returns a REFERENCE
                        ▼
┌────────────────────────────────────────────────────────┐
│ system prompt                                    [ 4%]  │
│ conversation history                              [18%]  │
│ { "rows": 5000, "page": 1, "page_size": 50,              │
│   "handle": "rs_abc123" }                         [ 1%]  │
│ ← agent pages through it, or fetches by handle           │
└────────────────────────────────────────────────────────┘

Instead of dumping the payload, the tool returns something small that describes the payload and gives the agent a way to work with it incrementally: a page of 50 rows plus a next_page cursor, a statistical summary (“5,000 rows, avg order value $42, top category: electronics”), or a handle/id the agent can pass to a second, narrower tool call — get_page(handle="rs_abc123", page=2) or get_rows(handle="rs_abc123", filter="status=failed"). The agent decides how much of the data it actually needs to reason about, and only that much crosses into context.

This is the same principle whether the tool is a SQL query, a search endpoint, a file-listing call, or a log fetcher: the tool’s job is to make a large result set usable, not to be a faithful pipe for all of it. Pagination, summarization, and handles are three different implementations of the same idea — shrink what crosses the boundary into the model’s context.

Why the other options fail

• A) Bigger context window treats the symptom, not the disease. Even a frontier-scale context window is still a finite, shared, and increasingly expensive budget — every extra token costs latency and money, and a bigger window just moves the wall back instead of removing it. The next query returns 50,000 rows and you’re back to the same failure, now with a bigger bill. It also does nothing for the model’s actual ability to reason over 5,000 unstructured rows — attention degrades over very long, undifferentiated spans of context long before you hit any hard token ceiling.
• C) Truncate randomly silently corrupts the data the agent thinks it’s reasoning over. If the model believes it has “the results” and it actually has an arbitrary 40% sample of them, every downstream conclusion — a sum, a “the top customer is X,” a “no records match” — can be quietly wrong, and there’s no signal to the agent that anything was cut. Silent data loss is worse than an honest limit; at least a has_more: true flag on a paginated response tells the truth.
• D) Call it twice doesn’t reduce the payload — it doubles the tool-call round trips and, if the second call returns the same 5,000 rows, doubles the context cost too. It doesn’t address the actual bottleneck, which is the size of a single response, not the number of times you fetch it.

The concept behind it

Zoom out from this one question and the pattern generalizes to a design rule worth internalizing: treat every tool response as a context-budget line item, not a free pipe. When you’re designing a tool that can plausibly return an unbounded or large result set — a search, a list, a query, a log tail — build the shaping mechanism into the contract from day one, because retrofitting it after agents are already depending on “give me everything” behavior is much harder.

There are three common shapes, and they solve slightly different problems:

A well-designed tool often supports more than one of these at once: return a summary by default, with a handle the agent can use to page into specifics only if it needs them. That default-small, opt-in-detail pattern keeps the common case cheap and the edge case still possible.

This connects directly to two other things worth understanding on the exam and in production. First, tool descriptions need to document the shaping behavior explicitly — “returns up to 50 rows per page; use next_page to continue” — because if the model doesn’t know pagination exists, it will never ask for the next page. Second, this is a context-engineering problem wearing a tool-design costume: the same discipline that says “don’t re-paste the same big block into every turn” (see prompt caching) also says “don’t let one tool call flood the budget you’re trying to protect.” A tool that returns 5,000 rows is functionally identical to a system prompt that’s 5,000 lines too long — both are unbounded content competing with the model’s ability to actually think.

One more nuance worth knowing: this isn’t only a raw-token problem. Even within a technically-fits-in-the-window response, a 5,000-row unstructured blob suffers from what’s sometimes called “lost in the middle” — the model’s attention to content buried in a huge, undifferentiated span degrades even when every token technically fit. Shaping the response isn’t just about staying under a hard limit; it’s about giving the model something it can actually reason over.

FAQ

Does this mean tools should never return large amounts of data? No — it means the tool decides the default shape, and the agent opts into more. A tool can absolutely support fetching all 5,000 rows across 100 paginated calls if the agent’s task genuinely requires it. The design rule is: don’t make the flood the only option, and don’t make it the default.

What’s the difference between pagination and a handle-based approach? Pagination assumes the agent will iterate through the whole set in order (page 1, 2, 3…). A handle is more flexible — it’s a reference the agent can use for arbitrary follow-up queries against the same result set (filter, sort, fetch just row 4,200) without re-running the original expensive query. Many production tools implement both: a handle for follow-up, paginated access under it.

Isn’t a bigger context window going to make this a non-issue eventually? Context windows have grown substantially, and that helps at the margins — but the underlying economics don’t disappear. Larger context costs more per call, latency scales with tokens processed, and the “lost in the middle” attention problem doesn’t go away just because the ceiling moved. Design for shaping regardless of window size; treat a big window as slack, not as permission to skip the discipline.

How does this show up in MCP specifically? An MCP server exposing a query or search tool should implement the same shaping in its tool result schema — page/cursor parameters, a has_more flag, or a resource handle the client can dereference. Because an MCP server may be shared across multiple agents and clients, getting this contract right once at the server level protects every consumer, rather than leaving each calling agent to invent its own truncation logic.

Not affiliated with, authorized, or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic. These are original practice questions for study, not real exam content — verify current exam details on Anthropic’s official pages.

Where this fits in the CCA series

This question sits in Domain 2 — Tool Design & MCP Integration, worth roughly 18% of the exam (as of mid-2026: Pearson VUE delivery, $125, retakes available, 60 questions in 120 minutes, pass threshold 720/1000 across five domains — verify the current numbers on Anthropic’s official pages before you sit the exam). If you haven’t already, start with the Claude Certified Architect (CCA) Exam Guide to see all five domains laid out on one diagram, and read MCP Explained on One Diagram for the mechanism behind why shared tools benefit from a standardized server contract.

Response shaping is one piece of a larger tool-contract picture. Pair it with Your Tool Returns a 500 — Then What? Designing a Tool Error Contract for the other half of “what a tool hands back,” 40 Tools, Wrong Picks, High Latency: Scoping Tools Per Agent for the input side of the same budget problem, and ‘Gets Data’ Is Too Vague: How to Write Tool Descriptions That Work for how to document pagination so the model actually uses it. Browse all tutorials for the rest of the series.