Context Engineering: What the Model Sees Is What You Design

June 23, 2026 · updated June 25, 2026 · The Hidden Cost of AI Coding (part 7)

▶ Watch on YouTube & subscribe to The Stack Underflow

If you remember 2022-era “prompt engineering,” you remember when the skill was essentially copywriting for robots — get the phrasing right, nail the tone, add a few examples. That made sense when a conversation was a single turn. It does not make sense when your agent is ten steps deep into a loop and your original prompt is a footnote in a 200,000-token window.

On June 25, 2025, Andrej Karpathy put a name to what practitioners had been discovering in production: “context engineering is the delicate art and science of filling the context window with the right information.” Not prompts. Not wording. Information architecture. By the time that post circulated, the idea had already been formalised independently by Lance Martin at LangChain, codified in ArXiv papers (2603.09619, 2604.04258), and — more importantly — validated by every serious team that had tried to build a multi-step agent and watched it gradually lose the plot.

The one-sentence version: Context engineering is the practice of deliberately designing everything the model sees when it decides — not just what you typed, but system instructions, tool schemas, retrieved docs, conversation history, tool outputs, and user memory.

Prompt Engineering vs. Context Engineering

The clearest way to see the difference is to put them side by side:

The moment AI moved from chat to agents — from one turn to many — the prompt stopped being the dominant input. By step five of an agent loop, your original prompt is a small fraction of what the model sees. Aubakirova et al. (2025) documented that average prompt length in agentic workflows grew fourfold between 2024 and 2025, from roughly 1,500 tokens to over 6,000. If you only optimize the wording, you optimize a thin slice of what actually drives the output.

You are not writing prompts anymore. You are curating the entire information environment the model operates in. The prompt is just one input among many.

Four Failure Modes (Why Contexts Break)

Context engineering is a named discipline because context fails in specific, repeatable ways. There are four of them:

1. Poisoning — Wrong information enters the context and corrupts subsequent outputs. The model keeps consulting the bad source because it is sitting right there in the window. Garbage in, garbage (confidently) out. This includes stale retrieved chunks, outdated memory entries, and tool results from a previous subtask that were never pruned.

2. Distraction — Too much accumulated content drowns out the real instruction. The underlying mechanism is well-studied: transformer attention scales quadratically with context length, so at 100,000 tokens the model is managing roughly 10 billion pairwise attention relationships. Each token’s normalized weight shrinks. Morph’s 2025 testing of 18 frontier models found 30-plus percent accuracy drops from the lost-in-the-middle effect — content positioned in the middle of a long context receives systematically less attention than content at the start or end. If you have seen a long agent session start ignoring your system prompt because 40,000 tokens of tool outputs have piled up on top of it, this is what happened.

3. Confusion — Ambiguous or overlapping content leaves the model unsure which source to follow. Two tool descriptions that do similar things. Two memory entries that say slightly different things. The model picks one arbitrarily, or hedges on every call.

4. Clash — Two pieces of context give directly contradictory instructions. The model has no internal tiebreaker. It cannot know which one wins. Neither can you, reliably.

Each failure mode has a fix. That is what makes this engineering rather than advice.

Four Levers (The Toolkit You Already Have)

Here is the useful surprise: if you have followed this playlist, you already know the fixes. Each episode covered one lever:

Failure Mode  →  Lever       →  What it does
──────────────────────────────────────────────────────────────
Poisoning     →  Select      →  Choose which tokens enter the
                               window in the first place
Distraction   →  Compress    →  Summarise long history to keep
                               signal-to-noise ratio high
Confusion     →  Structure   →  Order stable content before
                               variable content; cache hits follow
Clash         →  Isolate     →  Sub-agents get clean windows;
                               no context cross-contamination

Let’s unpack each with concrete mechanics:

Select — Choose which tokens belong in the window at all. Use a cheaper model (Claude Haiku 4.5 at $1/MTok input) for small classification tasks so you do not dump irrelevant context into a flagship’s window ($5/MTok for Opus 4.8). For retrieval, do not fetch everything — fetch what is relevant to this step, not the whole document. Minimum viable context is the goal.

Compress — Summarise long conversation history so it costs fewer tokens and keeps signal-to-noise ratio high. Raw history is expensive and distracting; a summary is cheap and focused. Preventive compaction — removing noise before it accumulates — outperforms post-hoc compression. The Morph (2025) data is stark: coding agent success rates decline measurably after 35 minutes of task time, and doubling task duration quadruples the failure rate.

Structure — Put stable content (system instructions, tool schemas) before variable content (user message, retrieved docs) so prompt caching actually hits. On Anthropic’s API, you can place up to four cache_control markers at different points in a request. Cache reads cost 0.1x the standard input price — 90 percent cheaper than re-processing the same tokens. For Claude Sonnet 4.6 at $3/MTok input, a cache read costs $0.30/MTok. Bad ordering (variable content first) kills your cache hit rate and turns a potential 90 percent discount into zero savings. Cache writes cost 1.25x for a 5-minute TTL or 2x for a 1-hour TTL — the investment pays back after two cache hits on a 5-minute cache, or after one hit on a 1-hour window (docs.anthropic.com, 2026).

Isolate — Break long tasks into sub-agents, each with its own clean context window, instead of letting one session accumulate state across dozens of steps. Context rot is the long-session equivalent of a memory leak: the model’s effective reasoning capacity degrades continuously, not all at once. Every frontier model tested by Chroma in 2025 showed the effect — none was immune.

Six episodes, four levers, one discipline.

ASCII View: The Context Window as a System

┌─────────────────────────────────────────────────────────┐
│  What the model sees at inference time                  │
│                                                         │
│  [System prompt / instructions]   <- STABLE (cache)    │
│  [Tool schemas]                   <- STABLE (cache)    │
│  [Retrieved documents]            <- VARIABLE          │
│  [Conversation history (compressed)] <- COMPRESSED     │
│  [Tool outputs]                   <- VARIABLE          │
│  [User message]                   <- VARIABLE          │
└─────────────────────────────────────────────────────────┘
        ^
  Context engineering = designing this entire box,
  not just the bottom line.

  Cache breakpoints go on the boundary between
  STABLE and VARIABLE content.
  Variable content above the breakpoint = zero cache hits.

The ordering matters for two reasons: cache efficiency (stable tokens at the top are written once and read many times) and attention salience (the model attends most strongly to the beginning and end of its context, so your system instructions survive better at the top).

How This Maps to Current Model Economics

The discipline is not abstract — it has direct dollar consequences. Here is what the current model tier looks like as of June 2026 (docs.anthropic.com):

The Select lever (routing small tasks to Haiku) gives you a 5x cost reduction versus Opus for the same token count. The Structure lever (caching system prompts and tool schemas) gives you a 90 percent reduction on every token that hits cache. Stack them and a well-engineered context can cost less than 10 percent of a naive one. That math is not hypothetical — it is what separates teams that can run agents in production from teams that cannot.

Harness Engineering: The Next Layer

The discipline is still naming itself. Some practitioners — and a growing body of 2025-2026 research (ArXiv 2604.08224) — are moving past “context engineering” to call the broader discipline harness engineering: managing not just the information the model sees, but the feedback loops, validation layers, safety rails, and orchestration logic around it.

The distinction is informational versus operational:

Context Engineering   =   Information Plane
                          (what tokens go in the window)

Harness Engineering   =   Control Plane
                          (what rules govern what the agent
                           can do with those tokens)

The model is one component of a system you design. Research from Microsoft’s Agent Framework (BUILD 2026) found that 65 percent of enterprise agent failures traced to harness defects — specifically context drift (the context drifts away from the original task goal), schema misalignment (tool schemas diverge from what the model was trained to expect), and state degradation (persistent state corrupts across sessions). None of these are model failures. All of them are harness failures.

Context engineering is the information-plane discipline. Harness engineering is the control-plane discipline. You need both.

How to Apply This Right Now

Concrete steps ordered by impact:

1. Audit your context order today. Open any agent you run and look at what comes first in the request. If your system prompt and tool schemas come after a large retrieved document block, you are paying full price for them on every call. Move them to the top and add a cache_control marker at the boundary. This change alone can cut costs by 50-80 percent on cache-eligible workloads.

2. Set a token budget per step, not per session. Decide the maximum context size for each step in your agent loop rather than letting it grow unbounded. When the window approaches the budget, compress history before adding new content, not after the model starts missing instructions.

3. Use model tiering deliberately. Route tasks by complexity: Haiku 4.5 for classification, extraction, and summarization; Sonnet 4.6 for reasoning and code generation; Opus 4.8 for long-horizon planning and complex debugging. The Select lever is the highest-leverage cost control you have.

4. Prefer subagents over long sessions. Any task that requires more than 10-15 turns of tool use is a candidate for decomposition. Give each subagent a focused window and a specific deliverable. The Isolate lever prevents context rot from compounding across a long task.

5. Monitor cache_read_input_tokens in every response. Anthropic’s API returns this field in the usage object. If it is consistently zero, your structure is wrong — stable content is not actually staying stable between calls. Fix the ordering before scaling the workload.

Common Misconceptions

“Better prompts fix most agent problems.” Not once the agent is multi-step. By step five, your prompt is a minority of the input. The fix is usually structural — what is in the window, in what order, from what sources. Rewording the system prompt rarely helps when the real issue is 40,000 tokens of irrelevant tool output sitting above it.

“Context engineering is just RAG.” Retrieval-Augmented Generation is one technique under the Select lever — you retrieve relevant chunks instead of passing the full document. Context engineering is the full discipline: Select, Compress, Structure, and Isolate together. RAG addresses poisoning and distraction partially; it does nothing about cache structure or subagent isolation.

“Bigger context windows solve the problem.” All 18 frontier models tested by Chroma (2025) showed performance degradation as context grew — regardless of window size. A 1M-token window does not eliminate context rot; it raises the ceiling at which rot becomes catastrophic. The engineering discipline is how you stay well below that ceiling on every call.

“The model handles contradictions internally.” It does not. When two instructions clash, the model has no deterministic way to resolve them. The behavior depends on which instruction appears earlier, which is phrased more authoritatively, and which domain the model associates with stronger training signal. You have to prevent the clash at the context-design stage.

Frequently Asked Questions

What is the practical difference between a “prompt” and “context”?

A prompt is the text you explicitly write as a user or developer instruction. Context is everything the model reads before it generates a token — system prompt, tool schemas, retrieved chunks, conversation history, tool call results, and your prompt. Prompt engineering tunes one of those inputs; context engineering designs all of them.

How do I know which failure mode is hitting my agent?

Look at the symptom: outputs that were correct and then degraded over the course of a session point to distraction or poisoning (the window filled with noise or bad data). Outputs that consistently pick the wrong tool or source point to confusion (overlapping tool descriptions or ambiguous retrieved content). Outputs that oscillate between two behaviors point to clash (contradictory instructions somewhere in the window). Compress or isolate for degradation over time; restructure tool schemas for confusion; audit contradictions for clash.

Is context engineering just for Claude, or does it apply to other models?

The four failure modes — poisoning, distraction, confusion, clash — are properties of transformer attention, not of any one model. The levers apply equally to GPT-5, Gemini, Llama, and any other context-window-based model. The specific caching mechanics differ per provider (Anthropic uses explicit cache_control markers; OpenAI uses automatic prefix caching on a longest-stable-prefix heuristic), but the discipline is universal.

How does prompt caching actually save money in practice?

On every request where stable content (system prompt, tool schemas, background documents) is identical to a previous request, the API returns those tokens from cache at 10 percent of the standard input price. On a workflow where a 5,000-token system prompt is sent with every call, caching converts $5.00/MTok on that block to $0.50/MTok for Opus 4.8. For a team sending 1,000 calls per day with a 5,000-token cached prefix, that is $22,500 saved per month on that one block alone. The investment is the cache write — 1.25x on the first call — which pays back after the second cache hit.

What comes after context engineering?

Harness engineering is the next layer — managing not just information but the feedback loops, validation layers, and safety rails around the model. Context engineering is the information-plane discipline; harness engineering is the control-plane discipline. The next playlist in this course, Agents at Scale, covers the full harness: orchestration patterns, failure recovery, observability, and the economics of running multi-agent systems in production.

Does context engineering change when using MCP servers?

Yes — it adds an extra surface to manage. MCP servers expose tools with their own schemas, and each tool schema occupies tokens in the context window. The naive pattern is to register every available MCP tool; the engineering discipline is to register only the tools relevant to the current task and use the Select lever aggressively. The 2026 MCP roadmap (modelcontextprotocol.io) is moving toward dynamic tool registration, which will make this easier — but the underlying principle of minimum viable context applies regardless.

Where This Fits in the Series

This episode is the finale of Playlist 3 of The Hidden Cost of AI Coding. The three playlists converge on a single job: building reliable, affordable, predictable AI systems. Context engineering is the name for most of the engineering that sits around the model — and the model, it turns out, is the easy part.

For the economic foundation, revisit AI Coding Tokens Explained and Why AI Coding Bills Explode. For the core failure mode this discipline prevents, see Context Rot Explained and Bigger Context Windows, Worse Memory. For the caching mechanics behind the Structure lever, see Prompt Caching: Anthropic vs OpenAI.

Once you have the information plane under control, the next playlist picks up on the control plane: Why AI Agent Projects Fail, Subagent Isolation and Context Rot, and Agent Observability cover the harness engineering layer where context discipline meets operational reality.

Browse all tutorials to see where this episode sits in the full course.