The Cheapest Way to Cut Agent Cost: Model Routing and Cheaper Loops

July 24, 2026 · Claude Certified Architect (CCA) Prep (part 13)

▶ Watch on YouTube & subscribe to The Stack Underflow

Your agent bill just tripled, and the reflex fix is always the same: strip out tools, cap max_tokens, or quietly downgrade to one cheaper model for everything and hope quality holds. All three feel like cost control. Only one of them actually is — and it isn’t the one most teams reach for first. The other two just move the pain around: one trades cost for broken output, the other trades cost for a dumber agent.

The one-sentence version: The cheapest way to cut agent cost without gutting quality is to route — send easy, routine steps to a small/cheap model and escalate only the genuinely hard steps to a large model, instead of paying frontier-model prices for every token of every step.

The question

You’re running an agentic system in production and the token bill is becoming a problem. You need to cut cost without visibly degrading the quality of what users get back. What’s the cheapest way to cut agent cost without gutting quality?

• A) One big model for everything
• B) Route: small model for easy/routing steps, escalate to a big model for hard steps
• C) Remove tools
• D) Lower max tokens to 1

Pause here and pick an answer before you scroll — the theory behind it matters more than the letter.

The answer

B. Route: use a small, cheap model for the easy and routine steps — classification, formatting, simple lookups, deciding which tool to call next — and escalate to a big model only for the steps that actually need frontier-level reasoning.

The mechanism is a router sitting in front of your model calls, not inside your prompt. Every step in the agent loop gets a difficulty judgment before it gets a model assignment:

                         ┌───────────────────────┐
   step arrives ───────► │   ROUTER (cheap call   │
                         │   or heuristic check)  │
                         └───────────┬───────────┘
                                     │
                    difficulty of THIS step?
                                     │
              ┌──────────────────────┴──────────────────────┐
              │ easy / routine                       hard / high-stakes
              ▼                                               ▼
     ┌─────────────────┐                             ┌─────────────────┐
     │  SMALL MODEL     │                             │  BIG MODEL      │
     │  (Haiku-class)   │                             │  (Opus-class)   │
     │  classify, route,│                             │  multi-step     │
     │  format, extract │                             │  reasoning,     │
     └─────────┬────────┘                             │  ambiguous case,│
               │                                       │  final judgment │
               ▼                                       └─────────┬───────┘
          cheap tokens                                            │
          spent here                                     $$$ spent only
                                                           where it matters

     $ SAVED = (steps routed cheap) × (price delta small vs big model)

The reason this beats “one big model for everything” isn’t a vague appeal to frugality — it’s that most steps inside an agent loop aren’t hard. Deciding which tool to call next, extracting a field from a JSON blob, checking a routine condition, or formatting a response are pattern-matching tasks a small model handles at near-identical quality to a frontier model, at a fraction of the per-token price. The expensive capability — deep multi-step reasoning, judgment calls under ambiguity, synthesizing conflicting signals — is exactly the capability you’re paying a premium for in a big model. Routing means you only buy that premium on the steps that need it. Quality doesn’t drop because the steps that get the small model were never the steps where model capability was the bottleneck.

This is the same principle tested from the other side in The Agent Loop Explained: an agent loop is a sequence of discrete think→act→observe steps, and once you see it that way, “which model handles this step” becomes a per-step decision instead of a system-wide setting.

Why the other options fail

• A) One big model for everything — This is the default, not a cost strategy. It pays frontier prices for every routing decision, every field extraction, and every trivial classification, on top of the steps that actually deserve it. It’s the baseline you’re trying to cut cost from, not a lever.
• C) Remove tools — Cutting tools doesn’t reduce cost per token; it reduces what the agent can do. An agent that can’t call the tools it needs either fails the task or falls back to guessing from its own training data — which is a correctness failure, not a cost optimization. Tool count and token cost are separate variables; conflating them fixes nothing and breaks capability.
• D) Lower max tokens to 1 — This caps output length, not the underlying cost driver, and at max_tokens: 1 the model literally cannot produce a usable response. It’s not a subtle quality tradeoff — it’s a system that stops functioning. Any real token-budget lever (shorter system prompts, tighter output schemas, summarized history) still has to leave the model room to actually answer.

The concept behind it

Model routing generalizes into a broader pattern worth carrying into every agentic system you design: match model capability to step difficulty, don’t apply a single capability level uniformly across a workflow. The same instinct shows up under different names — cascade routing, mixture-of-experts at the application layer, tiered inference — but the mechanism is always the same router-then-dispatch shape.

A practical routing policy usually layers a few signals, roughly in this order of cheapness to compute:

A few things matter if you actually build this:

1. The router itself should be cheap. If judging “is this step hard?” costs nearly as much as just running the big model, you’ve added latency without saving money. Simple heuristics or a cheap classification call usually beat a heavyweight router.
2. Escalation should be rare by design, not by accident. If half your steps escalate, your task decomposition is wrong, not your routing threshold. Revisit how the workflow is broken into steps — this is the same discipline covered in Your Agent Blew the Context Window, where decomposing a task correctly is what makes each piece tractable at a smaller scale.
3. Routing is a cost lever, not a correctness lever. It sits alongside, not instead of, the other techniques in this domain — batching for overnight jobs cuts cost differently (by trading latency for a bulk discount), and prompt caching cuts cost differently again (by not re-paying for the same context block on every call). A mature agentic system stacks all three: cache the shared context, route the model by step difficulty, and batch what doesn’t need to run in real time.
4. Don’t route on model size alone — route on step difficulty. The trap is treating “small model” and “big model” as one global switch. The exam framing, and the production reality, is the same: the routing decision happens per step inside the loop, because that’s the actual grain at which task difficulty varies.

The exam is testing whether you can tell the difference between a cost lever that preserves the capability you’re paying for and one that quietly deletes it. “One big model” over-pays. “Remove tools” and “cap output tokens” under-deliver. Routing is the only option on the list that changes where the money goes without changing what the agent can do.

FAQ

Does model routing actually save meaningful money, or is it marginal? It depends on the step mix, but in most agentic workflows the majority of steps — tool selection, field extraction, formatting, simple classification — are cheap-model-capable. If even 60–70% of steps route to a small model at a fraction of the per-token price of a frontier model, the blended cost drop is substantial, often the single biggest lever available before you touch architecture at all.

How do you decide the difficulty threshold for escalation? Start conservative: escalate anything ambiguous, and measure how often the small model’s output fails downstream validation or gets corrected by a human. Tune the threshold against that failure rate, not against a fixed rule written up front — the right threshold is workload-specific and drifts as your task distribution changes.

Is routing the same thing as using a smaller model everywhere for a “good enough” agent? No — that’s closer to option A run backwards, and it has the same flaw in the opposite direction: uniform capability applied regardless of step difficulty, just cheap instead of expensive. Routing’s value comes specifically from the mix — cheap where cheap suffices, expensive only where expense is earned.

Does routing add latency from the extra router call? It can, if the router itself is a full model call. In practice, most production routers use a heuristic (task type, input shape) or a very small/cheap classification step, which adds negligible latency compared to the savings — and even a genuine escalation round-trip is usually cheaper in total latency than running every step through a large model by default.

Not affiliated with, authorized, or endorsed by Anthropic. “Claude” and “Claude Certified Architect” are trademarks of Anthropic. These are original practice questions for study, not real exam content — verify current exam details on Anthropic’s official pages.

Where this fits in the CCA series

This is Domain 1 (Agentic Architecture & Orchestration), question 10 of 10 in the practice set — the last of the “how does the loop actually run” questions before the series moves into Domain 2’s tool-design territory. If routing and cost tradeoffs are new territory, back up to The Agent Loop Explained for the think→act→observe mechanics that every step-level decision (including routing) sits on top of, and to Your Agent Blew the Context Window for the decomposition discipline that keeps individual steps small enough to route cheaply in the first place. If you haven’t seen how MCP and RAG carve up an agent’s capabilities, RAG vs MCP and MCP Explained on One Diagram are worth the ten minutes.

For the full five-domain map of the exam — what’s tested, how it’s weighted, and where to start — see the Claude Certified Architect (CCA) Exam Guide. Browse all tutorials for the rest of the series.