How to Pin Model Output Format Using Few-Shot Examples

June 23, 2026 · updated June 25, 2026 · How Claude Actually Works (part 11)

▶ Watch on YouTube & subscribe to The Stack Underflow

You’ve already wired up structured output. The JSON comes back cleanly shaped. And yet — one field is still wrong. The European invoice total reads 123000 instead of 1234.56. The date flips from the seventh of April to the fourth of July. Nothing throws. No stop_reason of refusal — just silently, expensively incorrect data flowing downstream into your database.

Your first instinct is the temperature slider. Drag it to zero, lock it down, rerun. Same wrong answer, delivered now with maximum confidence. That instinct is the wrong mental model, and this episode corrects it. The fix is not a float between 0.0 and 1.0 — it is two or three example turns added to the messages array before your real request.

The one-sentence version: Two or three concrete input/output pairs pinned in the messages array teach the model your exact conventions more reliably than any combination of prose instructions or temperature adjustments.

The Bug: Silent, Expensive Format Failures

The canonical scenario from the video is a European invoice with two fields:

• Amount: 1.234,56 € — where . is a thousands separator and , is the decimal marker
• Date: 07/04/2026 — day-first, month-second (DD/MM/YYYY)

Feed that to a model with no guidance. The output will likely be:

Both errors are silent failures — they pass a type check, they are valid JSON, and they would slip through any validation that only checks schema shape. You would catch them in an audit or not at all. This is the kind of bug that costs real money in finance, logistics, and healthcare systems.

Why Temperature Zero Does Not Help

Temperature is a scalar applied during token sampling — the process by which the model picks the next token from its probability distribution over the vocabulary. At temperature 1.0, it samples proportionally from that distribution. At temperature 0.0, it always picks the single highest-probability token: a greedy argmax. Deterministic, yes. Correct? Only if the highest-probability token was already correct.

Temperature 1.0:          Temperature 0.0:
──────────────────         ──────────────────
token A: 40%  ←random     token A: 40%  ← always picked
token B: 35%
token C: 25%

Same wrong token A, just... always.

Temperature controls randomness, not knowledge. The model’s understanding of European number formatting — whatever it absorbed during pre-training — is baked into its weights. Those weights are frozen at inference time. You cannot reach into them with a slider. Lowering temperature just makes the model more committed to whatever answer those frozen weights already suggest, including a wrong one.

The training data is unreachable at runtime. The context window is yours.

The Real Fix: Few-Shot Examples in the Messages Array

The Messages API (docs.anthropic.com, 2026) accepts a messages array of alternating user and assistant turns. The model treats whatever appears before its next generation as the established conversation history. That means you can pre-populate the conversation with synthetic worked examples — and the model will generalize from them.

Here is the minimal fix for the invoice scenario:

[
  {
    "role": "user",
    "content": "Parse this amount: 2.000,00 €"
  },
  {
    "role": "assistant",
    "content": "{\"amount\": 2000.00, \"currency\": \"EUR\"}"
  },
  {
    "role": "user",
    "content": "Parse this date: 31/12/2025"
  },
  {
    "role": "assistant",
    "content": "{\"date\": \"2025-12-31\"}"
  },
  {
    "role": "user",
    "content": "Parse this invoice: Total 1.234,56 €, dated 07/04/2026"
  }
]

Same model. Same temperature (zero if you like — it still works either way). Output: {"amount": 1234.56, "currency": "EUR", "date": "2026-04-07"}. Two green checks.

The model generalized from your examples. It did not memorize the strings. It pattern-matched the input/output shape — European decimal comma maps to a standard float, DD/MM maps to ISO YYYY-MM-DD — and applied that pattern to the new input it had not seen before. That is few-shot prompting: demonstration-based behavioral steering without any weight updates.

Where Examples Live in the Request

The key architectural point: your examples live in the prompt, not in the training data. Training data is frozen. The prompt is yours to control at inference time. That means the fix is instant, costs only tokens, requires no GPU hours, and is fully reversible.

┌──────────────────────────────────────────────────────────┐
│  Runtime request to the Messages API                     │
│                                                          │
│  system: "You are a data parser."                        │
│                                                          │
│  messages:                                               │
│  ┌──────────────────────────────────────────────────┐   │
│  │  [0] user:      "2.000,00 €"   ← example 1 in   │   │
│  │  [1] assistant: {"amount":2000.00}  ← example 1 out │
│  │  [2] user:      "31/12/2025"   ← example 2 in   │   │
│  │  [3] assistant: {"date":"2025-12-31"} ← ex 2 out│   │
│  │  [4] user:      REAL REQUEST                     │   │
│  └──────────────────────────────────────────────────┘   │
│                                                          │
│  Training weights ─────────────── FROZEN, unreachable   │
└──────────────────────────────────────────────────────────┘

The model reads the entire messages array as its input context. The examples are not flagged as “special” — they look like prior turns of a real conversation. The model treats them as evidence of what correct behavior looks like in this context.

Show, Don’t Tell: Examples vs. Prose Rules

The temptation before reaching for few-shot prompting is to write a more detailed system prompt. Something like:

"When parsing European amounts, note that the period is a thousands
separator and the comma is the decimal marker. When parsing dates,
assume DD/MM/YYYY format unless otherwise indicated. Always output
ISO 8601 dates."

Compare that to two examples:

PROSE RULES (tell):            FEW-SHOT (show):
──────────────────────         ────────────────────────────────
"Use European decimal          user: 2.000,00 €
 conventions. Commas           assistant: {"amount": 2000.00}
 are decimal separators.
 Dates are DD/MM."             user: 31/12/2025
                               assistant: {"date": "2025-12-31"}
✕ still fails on: 1.234,56    ✓ handles: 1.234,56 correctly

Prose rules describe intent. Examples demonstrate the exact mapping. For edge cases and locale-specific conventions — the situations where the model’s training data is most ambiguous — demonstration reliably outperforms description. The model has seen thousands of English-language instructions about “decimal conventions” in training. Two concrete examples cut through that ambiguity directly.

The Dosage Curve and Prompt Caching

How many examples do you need? The accuracy curve rises steeply with one, two, and three examples, then flattens. Token cost climbs linearly the entire time.

Accuracy
  ^
  |       ●─────────── (plateau)
  |     ●
  |   ●
  | ●
  └─────────────────────────────► # examples
     1   2   3   4   5   6

Token cost: rises linearly the whole way

Two or three examples is the sweet spot. Piling on ten examples does not materially improve accuracy and consumes tokens that could carry useful context. The video is explicit on this — do not mistake more examples for better examples.

One cost optimization: because your examples are fixed across all requests (same strings, same order, at the top of the messages array), they are ideal candidates for prompt caching. The Anthropic API’s prompt caching feature (docs.anthropic.com, 2026) lets you mark a prefix of your input for caching. Cache writes cost 25% more than base input token price; cache reads cost 10% of base input price. If your three example turns total 300 tokens and you make 10,000 requests per day, the cache hit savings are substantial. Keep your examples stable and at the top of the messages array to maximize the cache hit rate. Episode 21 covers prompt caching mechanics in detail.

How to Apply This in Production

The practical workflow:

1. Identify the failure mode first. Run your real inputs through the model without examples. Collect the specific cases where output format is wrong. These become your example set.

2. Write examples that cover distinct surface forms. For the locale case: one with a large amount (thousands separator fires), one with a fractional amount (decimal fires), one with an ambiguous date. Diversity matters more than quantity.

3. Place examples as the first turns in messages. Not in the system prompt. The system prompt is for high-level behavioral framing; the messages array is where you pin specific input/output conventions.

4. Mark the example prefix for caching. If you are on a current Claude model (claude-opus-4-8, claude-sonnet-4-6, or similar from the Opus 4.x/Sonnet 4.x family — see the models overview at docs.anthropic.com for the current pinned snapshot IDs), use the cache_control field to cache the stable prefix.

5. Pair with forced tool calls or structured output. Few-shot examples pin format conventions; a tool call with a typed schema pins the output schema. Together, they are more reliable than either alone. Episode 10 covers forced tool calls.

6. Test edge cases explicitly. The value of few-shot prompting is precisely on edge cases. Write your evals around the failure modes, not the happy path.

Common Misconceptions

• “Temperature zero gives me deterministic, correct output.” It gives deterministic output — correct only if the model’s frozen weights already knew the right answer. Zero temperature amplifies the model’s existing confidence, not its accuracy. It will reproduce the same wrong answer with perfect consistency.

• “I need to fine-tune to teach the model my format conventions.” Fine-tuning rewrites model weights and is appropriate for persistent, broad behavioral shifts across all uses of a model. For per-request format pinning — specific locale rules, your JSON shape, your date convention — few-shot examples in the prompt are faster, cheaper, and reversible with no infrastructure.

• “More examples are always better.” Accuracy plateaus after two or three examples. Beyond that you are spending tokens for diminishing accuracy returns, and potentially pushing useful context out of the window entirely on long requests.

• “Prose instructions are equivalent to examples.” They are not. Prose describes a rule; examples demonstrate the exact mapping. For corner cases and locale-specific behavior, demonstration consistently outperforms description. The model has seen a great deal of natural-language rule-writing in training; it has seen far fewer precise worked examples of your specific convention.

Frequently Asked Questions

Does the model actually “learn” from few-shot examples? No, not in the training sense. No weights are updated. The examples become part of the input context. The model performs in-context pattern matching — it reads the example pairs, extracts the input/output mapping, and applies the same mapping to the new input. This is purely inference-time reasoning, not learning. The examples vanish when the request ends.

Can I mix few-shot examples with a system prompt? Yes, and you should. The system prompt sets high-level behavioral framing — “you are a data parser, always return valid JSON.” The few-shot examples in the messages array pin the specific edge-case conventions within that frame. They are complementary, not redundant. Put your worked examples as synthetic user/assistant turns in messages, not inside the system prompt itself.

What makes a good example set? Diversity of surface forms. If you are teaching locale handling, one example with a large round number, one with a small fractional amount, and one with an ambiguous date covers more of the input space than three nearly identical examples. If all your examples look the same, the model may overfit to superficial string features rather than generalizing the underlying mapping rule.

How does prompt caching interact with few-shot examples? If the same few-shot prefix appears at the start of every request — same examples, same order, same text — you qualify for a prompt cache hit on all but the first request. Cache reads cost roughly 10% of base input token price (docs.anthropic.com, 2026), turning a repetitive few-shot prefix from a recurring cost into a near-zero fixed cost. Keep your examples stable and early in the messages array to maximize cache utilization.

What if my few-shot examples conflict with the system prompt? The model resolves the conflict in favor of the most contextually specific signal. A concrete demonstration in the messages array usually wins over an abstract rule in the system prompt. If you see conflicts, make the system prompt rule and the example consistent. Do not use the system prompt to say “use ISO dates” while using examples that output a different format.

Does this technique work with all current Claude models? Yes. Few-shot prompting via the messages array is a core capability of the Messages API across all current models in the Claude 4.x family — including Opus 4.8, Opus 4.6, and Sonnet 4.6 (see the models overview at docs.anthropic.com for current pinned snapshot IDs). The technique is model-agnostic; it works because of how all autoregressive language models do in-context inference, not because of anything Claude-specific.

Where This Fits in the Series

This tutorial is part of How Claude Actually Works — a course that builds a mechanistic understanding of Claude from first principles. If you have not read the foundational episodes, The Claude Stack Mental Model gives the layered architecture this tutorial sits inside. How the Context Window Works explains why the messages array is your primary lever at inference time. Understanding stop_reason in the Claude API covers how to read the model’s response signals. Episode 12, Temperature, Top-P, and Top-K Explained, goes deep on the sampling mechanics that temperature actually controls — a useful companion if you want to fully understand why the temperature slider cannot do what few-shot examples do. Browse all tutorials to follow the full series in order.