How to Build a Structured Data Extraction Pipeline with Claude

June 23, 2026 · updated June 25, 2026 · How Claude Actually Works (part 30)

▶ Watch on YouTube & subscribe to The Stack Underflow

Document extraction in production is never just “send the PDF, get JSON back.” Real invoices carry European decimal formats, ambiguous dates, and the occasional field that is simply not there. A pipeline that works on your ten test samples will embarrass you on invoice 847. This tutorial assembles the exact five-technique stack the video demonstrates — forced schema, few-shot examples, validate-and-retry, confidence routing, and prompt caching — into one end-to-end flow you can actually ship.

The architecture sits on two planes: a prompts plane that carries policy, examples, and the output schema; and a reliability plane that extracts, validates, and routes every document. Understanding which technique belongs to which plane makes the whole thing far easier to debug and extend. These planes are part of the layered mental model covered in The Claude Stack; if the vocabulary is new, read that first.

The one-sentence version: Force the model into a typed schema with tool_choice, lock domain conventions with few-shot examples, retry once on validation failure, divert low-confidence rows to a human queue, and cache the stable prompt prefix so repeat invocations cost roughly 90% less.

The Five-Technique Stack at a Glance

Before diving into each technique, here is how they map to the two planes and what problem each one solves:

The Four-Stage Pipeline

A raw invoice enters on the left and traverses four stages before it either lands in the database or goes to a human review tray.

raw invoice
    │
    ▼
┌─────────┐     ┌──────────┐     ┌──────────┐     ┌──────────┐
│ EXTRACT │────▶│ VALIDATE │────▶│  ROUTE   │────▶│ DATABASE │
│ (tool)  │     │ (pydantic│     │ (≥0.70   │     │          │
│         │◀────│  model)  │     │  conf.)  │     │          │
│  retry  │     └──────────┘     └─────┬────┘     └──────────┘
└─────────┘                            │
                                       ▼ (< 0.70)
                                  human review tray

Each stage has a clear responsibility and a clear failure mode. Keeping them separate means you can swap out the validation model or the routing threshold without touching the extraction call.

Technique 1 — Force the Schema with Tool Use

The extraction stage uses tool_choice to force a single tool call. Claude is not asked to “write JSON” — it is required to call a named tool whose input is defined by a Pydantic model compiled into a JSON Schema. When this call succeeds, the API returns stop_reason: "tool_use" and the response content block is always a typed dict, never free-form prose.

from anthropic import Anthropic
from pydantic import BaseModel
from typing import Optional

client = Anthropic()

class InvoiceModel(BaseModel):
    vendor_name: str
    invoice_number: str
    total_amount: float
    currency: str
    invoice_date: str          # ISO 8601 — YYYY-MM-DD
    confidence: float          # 0.0–1.0, model self-assessed

# Pydantic v2 -> JSON Schema -> tool definition
invoice_tool = {
    "name": "extract_invoice",
    "description": "Extract structured fields from an invoice document.",
    "input_schema": InvoiceModel.model_json_schema(),
}

response = client.messages.create(
    model="claude-opus-4-8",          # claude-opus-4-8 as of June 2026
    max_tokens=1024,
    tools=[invoice_tool],
    tool_choice={"type": "tool", "name": "extract_invoice"},  # forced
    messages=messages,
)

# stop_reason will be "tool_use" — safe to index directly
extracted = response.content[0].input  # always a dict, never prose

tool_choice set to a specific tool name guarantees a structured output on every call. You are not parsing prose; you are reading a validated dict. This is fundamentally different from asking the model to “output JSON in your response” — that instruction produces prose that happens to look like JSON, and the model can add commentary outside the block whenever it feels helpful.

The model_json_schema() method is Pydantic v2. If you are on a mixed codebase, calling it on a v1 model raises AttributeError immediately — a fast way to surface the mismatch.

Technique 2 — Lock Conventions with Few-Shot Examples

Two prior user/assistant turns are pinned above every extraction call. One demonstrates the European decimal convention — 1.234,56 means twelve hundred thirty-four, not one point two. The other fixes the expected date output as ISO 8601.

FEW_SHOT_TURNS = [
    {
        "role": "user",
        "content": "Invoice shows total: 1.234,56 EUR",
    },
    {
        "role": "assistant",
        "content": [
            {
                "type": "tool_use",
                "id": "ex_001",
                "name": "extract_invoice",
                "input": {
                    "total_amount": 1234.56,
                    "currency": "EUR",
                    "confidence": 0.95,
                },
            }
        ],
    },
    {
        "role": "user",
        "content": "Invoice date field reads: 31/12/2025",
    },
    {
        "role": "assistant",
        "content": [
            {
                "type": "tool_use",
                "id": "ex_002",
                "name": "extract_invoice",
                "input": {
                    "invoice_date": "2025-12-31",
                    "confidence": 0.97,
                },
            }
        ],
    },
]

Few-shot examples are the cheapest way to enforce domain conventions without fine-tuning. They also serve as living documentation: a new engineer reading the prompt immediately sees why 1.234,56 maps to 1234.56 and why 31/12/2025 maps to 2025-12-31. Any convention the model cannot infer from generic training data belongs in a few-shot turn.

The placement matters: few-shot turns must appear before the live document in the message list. The model reads turns in order; examples it has not yet seen cannot influence the extraction it is about to do.

Technique 3 — Validate and Retry (Once)

After extraction, the output is run through the same Pydantic model. If validation fails, the error is appended as a new user turn and the model retries — exactly once.

from pydantic import ValidationError

def extract_with_retry(document: str) -> InvoiceModel:
    messages = [
        *FEW_SHOT_TURNS,
        {"role": "user", "content": document},
    ]

    for attempt in range(2):          # max one retry
        raw = call_extract(messages)  # calls the Messages API
        try:
            return InvoiceModel(**raw)
        except ValidationError as e:
            if attempt == 0:
                # Append the bad attempt + the error, let the model self-correct
                messages.append({
                    "role": "assistant",
                    "content": [{
                        "type": "tool_use",
                        "id": "retry_01",
                        "name": "extract_invoice",
                        "input": raw,
                    }],
                })
                messages.append({
                    "role": "user",
                    "content": (
                        f"Validation failed: {e}. "
                        "Please correct the output and call extract_invoice again."
                    ),
                })
            else:
                raise  # second failure → escalate

The one-retry ceiling is deliberate. A model that fails twice on the same document is almost certainly facing genuine ambiguity — something a human should resolve, not a third API call. Unbounded retries turn rare failures into runaway costs. The retry loop is a reliability mechanism, not a brute-force strategy.

Attempt 1:  EXTRACT ──▶ VALIDATE ──✕── append error ──┐
                                                        │
Attempt 2:  EXTRACT ──▶ VALIDATE ──✓── continue       ◀┘

Attempt 2 (still fails): raise ValidationError → escalate

Technique 4 — Confidence Routing

Every output row carries a confidence field (0.0–1.0). A threshold at 0.70 splits the stream:

def route(result: InvoiceModel) -> str:
    if result.confidence >= 0.70:
        write_to_database(result)
        return "auto-accepted"
    else:
        send_to_human_queue(result)
        return "routed-to-review"

This is the core insight of the pipeline: you do not need the model to be right on every document. You need it to know when it is uncertain and signal that uncertainty so your code can act on it. Humans see only the ambiguous slice, which is a far better use of their time than reviewing every extracted record.

The 0.70 threshold is a starting point. The right number depends on the cost of a bad auto-accept versus the cost of unnecessary human review in your domain. Measure the false-accept rate and the false-divert rate on a labeled sample before committing to a threshold.

Technique 5 — Cache the Stable Prefix

The vendor policy block — decimal rules, date rules, the “do not invent values” instruction, and the few-shot turns — does not change between invoices. Marking that prefix with a cache_control breakpoint means repeat invocations read from cache instead of reprocessing the same tokens on every call.

system_prompt = [
    {
        "type": "text",
        "text": VENDOR_POLICY + FEW_SHOT_TEXT,
        "cache_control": {"type": "ephemeral"},  # cache this prefix
    }
]

As of 2026, ephemeral is the only cache type. The default TTL is five minutes; you can extend it to one hour by passing the extended-cache-ttl-2025-04-11 beta header and specifying "ttl": "1h" inside the cache_control dict (docs.anthropic.com, 2026). Cache writes cost 125% of standard input tokens; cache reads cost 10% — so a single cache hit already pays for the write. On a thousand-invoice batch, that 90% reduction on input tokens is the difference between a pipeline that is economical and one that becomes a budget line item.

Cache is per-API-key and per-workspace. It does not persist across different callers or different keys. It is designed for high-throughput batches, not for sharing state across teams.

The Pipeline at Scale

Pour a thousand invoices through this pipeline and the numbers settle to roughly 92% auto-accepted into the database and 8% diverted to human review.

1,000 invoices
      │
      ▼
┌─────────────────────────────────────────┐
│  EXTRACT + FEW-SHOT + CACHED POLICY     │
│  (cached prefix → ~90% cheaper reads)  │
└────────────────────┬────────────────────┘
                     │
              ┌──────▼──────┐
              │   VALIDATE  │
              │  (pydantic) │
              └──────┬──────┘
                     │
         ┌───────────▼───────────┐
         │     ROUTE (≥ 0.70)    │
         └────────┬──────────────┘
                  │
      ┌───────────┴──────────────┐
      ▼                          ▼
 ~920 rows                   ~80 rows
 DATABASE                    HUMAN REVIEW
 (auto-accept)               (ambiguous slice)

The clean majority flows without human touch. The ambiguous slice — the genuinely hard cases — gets human attention. That is the leverage point of the whole architecture.

How to Apply This

To wire this up in your own project:

1. Define the schema first. Start with a Pydantic v2 model. Every field the downstream system will consume should be typed. Add a confidence: float field — the model will self-assess it in the same call.
2. Write the few-shot examples second. List the domain conventions that a model trained on generic English would get wrong. European number formats, fiscal-year dates, company-specific field names, and unit conventions are all candidates.
3. Build the retry loop third. Two attempts is the ceiling. Wire the ValidationError message back as a user turn on the first failure.
4. Set the confidence threshold empirically. Run the pipeline on a labeled sample. Measure false-accepts and false-diverts. Adjust the threshold until the human-review queue is a size your team can handle.
5. Add cache_control last. Once the policy block and few-shot turns are stable, mark the prefix. You will see the savings immediately on any batch that hits the same prefix twice.
6. Pin the model ID. Use a specific versioned model ID such as claude-opus-4-8 (docs.anthropic.com/en/docs/about-claude/models/overview, 2026). Evergreen aliases can silently change behavior on a model update.

Common Misconceptions

• “Forcing a schema is the same as asking for JSON in the prompt.” It is not. tool_choice with a specific tool name makes the model call a typed function; the API guarantees stop_reason: "tool_use" and a structured content block. A prompt instruction to “output JSON” still produces prose that happens to look like JSON — the model can add commentary outside the block when not constrained by tool use.
• “Few-shot examples are just for showing what the output looks like.” They also encode domain conventions that are invisible to a model trained on generic text. European number formatting, fiscal-year date conventions, and company-specific field names are all candidates for few-shot pins. The schema tells the model the shape; the examples tell it the meaning.
• “One retry is not enough — keep retrying until it passes.” Unbounded retries turn rare failures into runaway costs. If validation fails twice, you have hit the ceiling of what the model can self-correct given the document’s content. The right response is escalation to a human or a fallback path, not a third API call.
• “Confidence routing needs a separate classification step.” No — the confidence score is a field in the same tool_use output. The model produces it in a single call; your code reads it and branches. Adding a second model call for routing doubles latency and cost for no benefit.

Frequently Asked Questions

What Pydantic version works with model_json_schema()? Pydantic v2. In v1 the equivalent method is .schema(). If you are on a mixed codebase, calling model_json_schema() on a v1 model raises AttributeError immediately — a fast way to find the mismatch before it surfaces in production.

How do I choose the confidence threshold? Start by measuring the false-accept rate (confident rows that were actually wrong) and the false-divert rate (unconfident rows that were actually correct) on a labeled sample. 0.70 is a reasonable starting point, but the right number depends on the cost of a bad auto-accept versus the cost of unnecessary human review in your domain. For high-stakes documents like financial invoices, err toward a lower threshold and accept more human review.

Does cache_control: ephemeral persist across sessions or API keys? No. The cache is scoped to your API workspace and has a default TTL of five minutes of inactivity. As of February 2026, caching uses workspace-level isolation rather than organization-level isolation (docs.anthropic.com, 2026). The one-hour TTL extension is available via the extended-cache-ttl-2025-04-11 beta header. Neither option shares state across different workspaces or keys.

Can I use this pattern for documents other than invoices? Yes. The same five techniques apply to any document type where you need typed fields, domain conventions, and a human-in-the-loop fallback: receipts, contracts, medical forms, shipping manifests, and insurance claims all fit this shape. The Pydantic schema and the few-shot examples change; the pipeline structure stays the same.

When should I use this pipeline versus the native structured outputs API? The native output_config.format path (generally available as of 2025 for Claude Sonnet 4.5, Opus 4.5, and Haiku 4.5) is simpler for straightforward schemas with no retry or routing logic. Use the tool-use pipeline when you need the confidence field embedded in the same output, the retry loop feeding the error back as a conversation turn, or the full prompt-caching setup — because cache_control works on system prompt blocks and tool definitions, not on output_config schemas.

What model should I use for a high-volume extraction batch? For cost-sensitive batches with well-structured schemas and a solid few-shot set, claude-sonnet-4-6 offers frontier-level extraction at a lower per-token cost than Opus. Reserve claude-opus-4-8 for schemas with many optional fields, highly ambiguous documents, or where extraction quality directly affects downstream financial decisions (docs.anthropic.com/en/docs/about-claude/models/overview, 2026).

Where This Fits in the Series

This tutorial is the sixth applied scenario in How Claude Actually Works — the capstone of the course’s reliability arc. It draws on four earlier episodes: tool use mechanics, prompt caching, confidence fields and human-in-the-loop routing, and few-shot prompting. If any of those pieces feel unfamiliar, those tutorials fill the gap before you wire the full pipeline.

The series finale — the next episode — steps back from individual techniques and maps the entire Claude Stack as one integrated architecture. Browse all tutorials to follow the full series.