How to Guarantee JSON Output from Claude with Structured Outputs

June 23, 2026 · updated June 25, 2026 · How Claude Actually Works (part 10)

▶ Watch on YouTube & subscribe to The Stack Underflow

Every production system that talks to an LLM eventually hits the same wall: the model was asked to return JSON, it returned JSON ninety-nine times, and on the hundredth it prefaced its response with “Sure! Here you go:” and wrapped everything in a markdown code fence. Your parser threw an exception at 3 a.m. and nobody was watching.

The failure is not a model quality issue you can fix by rephrasing your prompt. It is a structural property of autoregressive generation: every token is sampled from a probability distribution, and sometimes the highest-probability token is S — for “Sure.” The only reliable fix is to eliminate the model’s ability to choose prose over structure in the first place. This tutorial shows you exactly how to do that, using the forced tool-call mechanism and Claude’s native structured output API.

The one-sentence version: Instead of prompting the model to “return JSON,” force it to fill in a typed tool call — or use Claude’s native output_config.format — so the shape of the response is enforced by the API, not left to the model’s discretion.

Why polite prompting is a probabilistic contract

Autoregressive generation is the process by which a language model predicts one token at a time, each conditioned on everything before it. The model has no separate “mode” for structured versus unstructured output. When you write "respond ONLY with JSON", the model weights that instruction heavily — but it does not make prose output impossible. It makes it less likely. Less likely is not the same as impossible.

Fire the same prompt three times with temperature 0.7:

Attempt 1: {"invoice_number": "INV-001", "vendor": "Acme", "total": 142.50}
Attempt 2: {"invoice_number": "INV-001", "vendor": "Acme", "total": 142.50}
Attempt 3: Sure! Here you go:
           {"invoice_number": "INV-001", "vendor": "Acme", "total": 142.50}
           (wrapped in a markdown ```json fence — not raw JSON)

Your json.loads() on attempt 3 raises a JSONDecodeError. Setting temperature to 0 makes the output deterministic given a fixed prompt, but the deterministic output can still be a preamble followed by a JSON fence. Temperature controls randomness; it does not control format. You need a structural constraint.

The core pattern: schema as tool

The insight behind forced tool calls is that a tool’s input_schema is already a JSON Schema — exactly the structure you want your output to conform to. Instead of asking the model to produce JSON and then hoping it does, you describe your desired output shape as a tool’s input, then force the model to call that tool.

Your desired output shape
        │
        ▼
  Pydantic model → JSON Schema → tool input_schema
                                        │
                          tool_choice: {type:"tool", name:"extract_invoice"}
                                        │
                                        ▼
                         Model MUST emit a tool_use block
                         (prose output is structurally impossible)
                                        │
                              ┌─────────┴─────────┐
                           valid                invalid
                              │                    │
                        typed object          ValidationError
                                                   │
                                          append error as user turn
                                          retry once (hard ceiling)

The key move is tool_choice with type: "tool" and an explicit name. When this is set, the API prefills the assistant turn to force a tool call — the model cannot emit a natural-language reply first. The free-text escape hatch is closed at the API level, not the model level.

Step-by-step: building the forced tool-call pipeline

Step 1 — Define your contract as a Pydantic model

Pydantic is a Python data-validation library that also generates JSON Schema from Python type annotations. Define every field your downstream code needs, with exact types. Mark fields that genuinely may be absent as Optional with a default of None:

from pydantic import BaseModel
from typing import Optional

class Invoice(BaseModel):
    invoice_number: str
    vendor: str
    total: float
    po_number: Optional[str] = None   # absent in some invoices

Required fields are required. Optional fields return null when absent — the model does not invent a value to fill the gap. This is the schema preventing hallucination in a way that prompt wording cannot guarantee: the schema is a hard constraint, not a polite request.

Step 2 — Derive the tool definition from the schema

Convert the Pydantic model into a tool definition. The tool’s input_schema is the JSON Schema representation of your model. Name the tool descriptively — extract_invoice — so the model understands its purpose:

import json

extract_invoice_tool = {
    "name": "extract_invoice",
    "description": "Extract structured invoice fields from the provided text.",
    "input_schema": json.loads(Invoice.model_json_schema(mode="serialization").model_dump_json())
}

The model’s job has now changed. It is no longer “write text that looks like JSON.” It is “fill in the arguments for this tool call.” That is a completely different generative task.

Step 3 — Force the call

Pass tool_choice with type: "tool" and the tool name. As of the current Messages API (docs.anthropic.com, 2025), the four valid tool_choice types are auto, any, tool, and none. Only tool forces a specific named tool:

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-4-8",          # or claude-sonnet-4-6, claude-haiku-4-5
    max_tokens=1024,
    tools=[extract_invoice_tool],
    tool_choice={"type": "tool", "name": "extract_invoice"},
    messages=[{"role": "user", "content": raw_invoice_text}],
)

# Response always contains a tool_use block — never prose
tool_block = next(b for b in response.content if b.type == "tool_use")
raw_args = tool_block.input   # already a dict, no json.loads needed

The stop_reason on this response will be "tool_use" — the model stopped because it produced the required tool call, not because it chose to stop generating. If you see any other stop_reason here, something is wrong with the request.

Step 4 — Validate gate with a hard retry ceiling

The forced call guarantees the output is a tool-use block, but the model can still populate fields with structurally wrong types (a string where a float is expected, for example). Run the raw arguments through Pydantic again:

from pydantic import ValidationError

MAX_RETRIES = 1   # hard ceiling — do not raise this
messages = [{"role": "user", "content": raw_invoice_text}]
invoice = None

for attempt in range(MAX_RETRIES + 1):
    response = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=1024,
        tools=[extract_invoice_tool],
        tool_choice={"type": "tool", "name": "extract_invoice"},
        messages=messages,
    )
    raw_args = next(b for b in response.content if b.type == "tool_use").input

    try:
        invoice = Invoice(**raw_args)
        break
    except ValidationError as e:
        if attempt >= MAX_RETRIES:
            raise   # give up and surface the error
        # paste the error back as a new user turn
        messages.append({"role": "assistant", "content": response.content})
        messages.append({
            "role": "user",
            "content": f"Your previous output failed validation: {e}. Try again."
        })

The ceiling of one retry is not arbitrary conservatism — it is production hygiene. A bad document (blurry scan, ambiguous layout, missing fields) will fail validation repeatedly. Without a ceiling, one bad document becomes twenty API calls and a surprise bill. If two attempts fail, the document is a human problem, not a model problem.

The modern path: native structured outputs

As of November 2025 (generally available), Claude has native structured outputs that go deeper than the tool-call pattern. Instead of constraining the model by forcing a tool call, the API compiles your JSON Schema into an inference-time grammar that makes it physically impossible for the model to emit tokens that violate the schema during generation. This is called grammar-constrained generation or constrained decoding.

response = client.messages.create(
    model="claude-sonnet-4-6",   # or claude-opus-4-8, claude-haiku-4-5
    max_tokens=1024,
    output_config={
        "format": {
            "type": "json_schema",
            "schema": Invoice.model_json_schema()
        }
    },
    messages=[{"role": "user", "content": raw_invoice_text}],
)

import json
invoice_data = json.loads(response.content[0].text)
invoice = Invoice(**invoice_data)

The response comes back as a text block whose content is guaranteed valid JSON matching the schema. No tool wrapper, no tool_use block parsing, no stop_reason checking — just clean JSON that you validate with Pydantic as a final sanity gate.

The old output_format parameter (from an earlier beta) is deprecated — remove it from any code that still uses it.

Optional fields vs. hallucination: a subtle guarantee

When a field is marked Optional in the schema (JSON Schema: "required" array omits the field name, or the type includes "null"), the model returns null for absent values rather than fabricating a plausible-looking one. This is the schema doing work that prompting cannot do reliably.

Invoice document with no PO number:

  Forced tool call with Optional[str] po_number:
  → {"invoice_number": "INV-001", "vendor": "Acme", "total": 142.50, "po_number": null}

  Polite prompt "do not hallucinate":
  → {"invoice_number": "INV-001", "vendor": "Acme", "total": 142.50, "po_number": "N/A"}
    (model invented "N/A" — not null, not an error, silently wrong)

The difference matters when your downstream code checks if invoice.po_number is None. Hallucinated values pass that check and corrupt your records silently. Schema-constrained null values fail the check correctly.

Common misconceptions

• “A better system prompt will fix the format problem.” A well-crafted system prompt reduces format failures. It cannot eliminate them. The model’s output is always a probability distribution over tokens; a very good prompt shifts that distribution toward valid JSON but never collapses it to a point mass on valid JSON. Structural enforcement is a categorical fix; prompt wording is a probabilistic one.

• “Temperature 0 guarantees consistent output format.” Temperature 0 makes the output deterministic given the exact same prompt and model version. The deterministic output can still be a sentence followed by a JSON fence. Determinism and format conformance are orthogonal properties.

• “I should retry until it passes validation.” An unbounded retry loop converts a bad document into an unbounded API bill and an unresponsive pipeline. One retry gives the model a chance to self-correct on a transient mistake. If it fails twice, the input is the problem — hand it to a human or log it for manual review.

• “The tool-call pattern is just a workaround for the real JSON mode.” The forced tool-call pattern is not a hack — it is the mechanism that the native structured output feature builds on conceptually. Understanding it gives you the vocabulary to debug constrained generation failures, evaluate schema design tradeoffs, and adapt the pattern to environments where output_config.format is not yet available.

Frequently asked questions

What is the difference between tool_choice: "auto" and tool_choice: {type: "tool", name: "..."}? With auto, the model decides each turn whether to call a tool or reply with text. With the explicit name form, the model is forced to call that specific tool and cannot reply with prose. For structured-output use cases you almost always want the explicit name form. The any variant forces the model to call one of the provided tools but does not specify which one — useful when you have multiple extraction tools and the model should pick the right one.

Does output_config.format work with all Claude models? As of June 2026, native structured outputs via output_config.format are supported on Claude Opus 4.x, Claude Sonnet 4.x, and Claude Haiku 4.5 and later. The frontier model Claude Fable 5 also supports it. Older pinned model IDs (pre-2025) may not support the feature. Check the Anthropic API release notes (docs.anthropic.com/en/release-notes/api) for the exact model version floor.

What happens if a required field genuinely does not exist in the document? With a required field in the schema, the model will attempt to populate it. If there is no value to extract, it may hallucinate a plausible one. This is why designing your schema correctly matters: fields that are genuinely absent in some documents should be Optional. The validation gate will then catch structurally invalid outputs (wrong type), but schema design is your defense against hallucinated-but-valid-looking values in required fields.

Should I use forced tool calls for all LLM calls? Only when you need a specific structured shape that downstream code will parse programmatically. For open-ended generation, summarization, code writing, or conversation, forcing a tool call adds API overhead and removes the model’s ability to express nuance. Reserve structural enforcement for extraction, classification, and routing tasks where output shape is a hard contract.

How do I choose between the tool-call approach and output_config.format? For new production code: use output_config.format. It is cleaner, has no tool overhead, and the grammar-constrained decoding is more robust. For debugging structured-output failures or working with older model versions: understand the forced tool-call mechanism — it makes stop_reason, tool_use block parsing, and retry logic explicit, which helps you reason about what went wrong. The two are pedagogically complementary.

Can I combine tool_choice: {type:"tool"} with strict: true? Yes. Setting strict: true on a tool definition tells the API to enforce the tool’s input_schema strictly during tool-call generation, similar to how output_config.format constrains text output. This is the forced tool-call approach with an extra layer of schema enforcement layered in. As of 2025, this is supported on current Claude models (docs.anthropic.com).

Where this fits in the series

This tutorial sits in the Prompts and Structured Output plane of the Claude Stack mental model — a cross-cutting layer that shapes what comes out of the model. The mechanism depends on the tool-call machinery explained in How Claude uses tools and the stop_reason values covered in Understanding stop_reason in the Claude API. The invoice extractor built here is the capstone project for the series, assembled fully in Structured data extraction pipeline. For the complementary approach — pinning model behavior through examples rather than schema constraints — see Pin model behavior with few-shot examples. Browse all tutorials to follow the full series.