Structured Data Extraction with LLMs: A Practical Engineering Guide

Structured extraction is the process of turning unstructured text, PDFs, emails, HTML, transcripts, into typed, validated records your database or downstream service can actually use. Large language models have made this dramatically easier than the regex-and-NER pipelines engineers built for a decade, because a single prompt plus a schema can replace hundreds of hand-written extraction rules. This guide covers how structured extraction actually works under the hood, how to design schemas that hold up in production, and the failure modes that show up once you move past a demo.

What Structured Extraction Actually Means

Structured extraction takes free-form input and produces output that conforms to a known shape: a JSON object, a set of database rows, a typed class instance. The contract is the schema. Before LLMs, engineers built this with three tools: regex for pattern matching, named entity recognition (NER) models for tagging spans of text, and rule-based parsers for known document layouts (think invoices from a fixed vendor). Each of those approaches breaks the moment the input format shifts even slightly.

LLM-based structured extraction works differently. You give the model a schema (a JSON Schema, a Pydantic model, a TypeScript interface) and the source text, and the model returns data that matches that schema. Because the model has general language understanding, it handles format drift, synonyms, and missing fields far better than pattern matching. The tradeoff is nondeterminism: the same input can produce slightly different output across calls unless you constrain the model carefully. That constraint is the entire engineering problem this article is about.

Function Calling and Tool Use: The Core Extraction Pattern

The reliable way to do structured extraction with a modern LLM is not to ask it to "output JSON" in a prompt and hope. It's to use the model's native tool use (also called function calling) feature, where you define a schema as a tool the model must call, and the API enforces that the response matches that schema's shape.

Here's the pattern using the Anthropic Python SDK. The model is given a single tool definition that describes the exact fields we want extracted from a support ticket:

import anthropic

client = anthropic.Anthropic()

extraction_tool = {
    "name": "record_ticket",
    "description": "Record structured fields extracted from a support ticket",
    "input_schema": {
        "type": "object",
        "properties": {
            "customer_name": {"type": "string"},
            "issue_category": {
                "type": "string",
                "enum": ["billing", "bug", "feature_request", "account_access"]
            },
            "severity": {"type": "string", "enum": ["low", "medium", "high"]},
            "summary": {"type": "string"},
            "requires_followup": {"type": "boolean"}
        },
        "required": ["customer_name", "issue_category", "severity", "summary"]
    }
}

def extract_ticket(ticket_text: str) -> dict:
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        tools=[extraction_tool],
        tool_choice={"type": "tool", "name": "record_ticket"},
        messages=[{"role": "user", "content": ticket_text}]
    )
    for block in response.content:
        if block.type == "tool_use":
            return block.input
    raise ValueError("Model did not return a tool call")

Forcing tool_choice to the specific tool name matters. Without it, the model can choose to respond in plain text instead of calling the tool, and you lose the structural guarantee. This is the single most common bug in first-draft extraction code: the happy path works in testing, then one weird input causes the model to explain itself in prose instead of returning structured data, and the pipeline throws a parsing error in production.

Defining Extraction Schemas with Pydantic

Raw JSON Schema is verbose to write and easy to get subtly wrong (missing a required field, mistyping an enum). In Python, define your schema as a Pydantic model instead, and generate the JSON Schema from it. This gives you validation on the way out as well as a schema definition on the way in.

from pydantic import BaseModel, Field
from typing import Literal
from enum import Enum

class IssueCategory(str, Enum):
    billing = "billing"
    bug = "bug"
    feature_request = "feature_request"
    account_access = "account_access"

class SupportTicket(BaseModel):
    customer_name: str
    issue_category: IssueCategory
    severity: Literal["low", "medium", "high"]
    summary: str = Field(description="One sentence summary of the issue")
    requires_followup: bool = False

schema = SupportTicket.model_json_schema()

You can pass schema straight into the tool definition's input_schema field. When the model returns a tool call, run the raw dict through SupportTicket.model_validate(result) before you touch it anywhere else. This step catches the cases where the model technically matched the JSON shape but put a string where an enum was expected, or left a required field as an empty string instead of a proper value. Never trust model output as typed data until it has passed through your own validator, even when you used tool use to constrain the model.

A library like instructor wraps this whole loop (call the model, parse the tool call, validate against Pydantic, retry with the validation error fed back to the model on failure) so you don't write it by hand for every project. It's worth adopting once you have more than two or three extraction schemas in a codebase, since the retry-with-error-feedback loop is easy to get wrong the first few times you write it yourself.

Structured Extraction for Nested and Repeated Fields

Real documents rarely map to a flat schema. An invoice has a list of line items. A resume has a list of jobs, each with a list of bullet points. A contract has nested clauses with cross-references. LLMs handle nested and repeated structures well as long as your schema makes the nesting explicit rather than implicit.

class LineItem(BaseModel):
    description: str
    quantity: int
    unit_price: float
    line_total: float

class Invoice(BaseModel):
    vendor_name: str
    invoice_number: str
    invoice_date: str
    line_items: list[LineItem]
    subtotal: float
    tax: float
    total: float

Two things to watch for with nested schemas. First, arithmetic: models are language predictors, not calculators, and asking a model to compute subtotal + tax = total inside the extraction call is asking for trouble on documents with many line items. Extract the raw numbers as they appear in the source text, then recompute derived totals in your own code and flag a mismatch as a data quality signal rather than silently trusting whatever the model wrote for total.

Second, cardinality drift: on a long document, models sometimes truncate a list (returning 8 line items when there were 12) if the output is getting long relative to max_tokens. Set max_tokens generously for any schema with an unbounded array field, and add a sanity check that counts extracted line items against a rough count from a cheap regex pass over the source text, so you catch truncation instead of silently shipping partial data.

Handling Long Documents: Chunking and Extraction Merging

Most extraction targets, contracts, transcripts, research papers, don't fit comfortably in a single call once you account for the input length plus a reasoning budget plus the output schema. Two strategies handle this: extend the extraction across chunks and merge, or use a two-pass approach.

For extraction where each field maps to roughly one location in the document (an invoice number, a signature date), a two-pass approach works well: first pass does document classification and section-finding (cheap, small output), second pass runs targeted extraction only on the relevant section.

For extraction where the target is repeated across the whole document (line items across a 40-page purchase order, action items across a two-hour meeting transcript), chunk the document and extract from each chunk independently, then merge:

def chunk_text(text: str, chunk_size: int = 6000, overlap: int = 500) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

def extract_all_line_items(document_text: str) -> list[LineItem]:
    all_items = []
    for chunk in chunk_text(document_text):
        result = extract_ticket(chunk)  # swap for your line-item extractor
        all_items.extend(result.get("line_items", []))
    return dedupe_line_items(all_items)

The overlap window matters: without it, a line item that straddles a chunk boundary gets split and neither chunk extracts it cleanly. The dedupe_line_items step matters just as much, since the overlap region will produce the same item twice from adjacent chunks. A simple dedupe keyed on (description, quantity, unit_price) catches most duplicates; for noisier text, a normalized string similarity check on the description field is worth the extra code.

Validation, Retries, and Self-Correction

Even with tool use forcing schema-shaped output, validation failures happen: an enum value that's close but not exact, a date in the wrong format, a required field the model left null because the source text genuinely didn't contain it. Build a retry loop that feeds the validation error back to the model rather than just retrying the same prompt blind.

from pydantic import ValidationError

def extract_with_retry(ticket_text: str, max_attempts: int = 3) -> SupportTicket:
    messages = [{"role": "user", "content": ticket_text}]
    for attempt in range(max_attempts):
        response = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            tools=[extraction_tool],
            tool_choice={"type": "tool", "name": "record_ticket"},
            messages=messages
        )
        tool_call = next(b for b in response.content if b.type == "tool_use")
        try:
            return SupportTicket.model_validate(tool_call.input)
        except ValidationError as e:
            messages.append({"role": "assistant", "content": response.content})
            messages.append({
                "role": "user",
                "content": [{
                    "type": "tool_result",
                    "tool_use_id": tool_call.id,
                    "content": f"Validation failed: {e}. Please correct and call the tool again.",
                    "is_error": True
                }]
            })
    raise ValueError(f"Extraction failed validation after {max_attempts} attempts")

Feeding the actual Pydantic error message back to the model, rather than a generic "try again," is what makes this loop converge in one or two retries instead of looping until it hits max_attempts. Models are good at parsing their own validation errors when you give them the exact field and constraint that failed.

Structured Extraction vs Traditional NER and Regex

Regex and rule-based parsers are still the right tool when the input format is genuinely fixed: extracting a phone number pattern, parsing a well-known CSV export, pulling fields from a single vendor's invoice template that never changes. They're deterministic, fast, and free to run at scale. Don't replace a working regex with an LLM call just because LLMs are the current trend; you'll add latency and cost for no accuracy gain.

Classic NER models (spaCy pipelines, fine-tuned BERT-style taggers) still win when you need to tag thousands of documents per second at very low cost and the entity types are well-defined and stable (person names, organizations, dates). They also don't have the nondeterminism problem.

LLM-based structured extraction wins when the input format varies (emails from different customers, resumes in different layouts, contracts from different law firms), when you need semantic judgment as part of the extraction (classifying severity, summarizing intent, inferring a category that isn't literally stated in the text), or when building and maintaining a rule-based parser for each format variant would cost more engineering time than the LLM API calls. In practice, most production pipelines end up as a hybrid: cheap deterministic extraction for the fields that are truly fixed-format, LLM extraction for the fields that require judgment or handle format variance.

Evaluating Extraction Quality in Production

You cannot ship a structured extraction pipeline without a held-out evaluation set, because the failure modes only show up on the long tail of real documents, not the three examples you tested with during development. Build an eval set of 50-200 real documents with hand-labeled ground truth, and score every schema change or prompt change against it before shipping.

Score at the field level, not just "did the whole record match." A pipeline that gets customer_name right 99% of the time but severity right 80% of the time has a specific, fixable problem; an aggregate accuracy score hides that. For each field, track exact match rate for categorical and numeric fields, and a fuzzy match (edit distance or embedding similarity) for free-text fields like summaries, since two reasonable summaries of the same ticket will never be byte-identical.

def score_field_accuracy(predictions: list[dict], ground_truth: list[dict], field: str) -> float:
    correct = sum(
        1 for p, g in zip(predictions, ground_truth)
        if p.get(field) == g.get(field)
    )
    return correct / len(ground_truth)

Re-run this eval whenever you change the model version, the prompt, or the schema. Model upgrades in particular can shift extraction behavior in ways that pass your quick manual spot-check but regress a specific field across the full eval set, so don't skip the eval just because a model swap "seems like an upgrade."

Common Pitfalls in Structured Extraction Pipelines

Trusting output without validation. Tool use constrains the shape of the response, not its correctness. A model can return a syntactically valid JSON object with a hallucinated value in a required field. Always validate against your schema and add domain checks (a date field with a value 200 years in the future should fail, even though it passes a basic type check).

No fallback for missing information. If the source text doesn't contain a field, models sometimes hallucinate a plausible-looking value instead of returning null. Make optional fields explicitly optional in your schema, and instruct the model directly to leave a field empty rather than guess, then check for suspiciously "too clean" outputs on inputs you know are missing that data.

Ignoring cost at scale. A single extraction call is cheap. A million-document backfill running the same extraction repeatedly is not. Cache extraction results keyed on a content hash so you never re-extract an unchanged document, and consider a cheaper/smaller model for a first-pass filter (is this even the right document type?) before running the full extraction schema on every input.

Schema sprawl. Adding "just one more field" to a shared extraction schema over months turns a focused, reliable extractor into a bloated one that has to reason about twenty fields at once, which measurably increases error rates on all of them. Split large schemas into a few focused extraction calls rather than one call trying to fill in forty fields.

No monitoring for drift. Extraction accuracy silently degrades when the input distribution shifts, a new document template shows up, a customer starts writing tickets in a different language, without any code changing. Log a sample of low-confidence or validation-failed extractions to a review queue so a human catches the drift before it becomes a data quality incident.

Building a Production Structured Extraction Pipeline

Put the pieces together into a pipeline shape that holds up under real traffic:

1. Classify and route. A cheap first pass determines document type and routes to the right schema. Don't run every document through every schema.
2. Chunk if needed. Apply chunking with overlap only for documents that exceed a safe single-call size; skip it for short documents to save latency.
3. Extract with tool use. Force tool_choice to the specific extraction tool. Never rely on freeform JSON parsing from a text response.
4. Validate. Run every result through your Pydantic model (or equivalent) before it touches a database.
5. Retry with error feedback. On validation failure, feed the exact error back to the model, capped at 2-3 attempts.
6. Recompute derived fields. Don't trust the model's arithmetic; recompute totals, counts, and cross-field checks in your own code.
7. Log low-confidence and failed extractions to a review queue instead of silently dropping or force-inserting bad data.
8. Re-run the eval set on every prompt, schema, or model change before deploying.

None of these steps are exotic engineering. The pipeline is the same shape as any other data validation pipeline you've built, with an LLM call standing in for what used to be a hand-written parser. The engineering discipline, schema-first design, validation at every boundary, held-out eval sets, is what makes the difference between a demo that works on three examples and a pipeline that survives contact with a million real documents.

FAQ

What's the difference between structured extraction and function calling? Function calling (tool use) is the API mechanism; structured extraction is the use case. You use function calling to force a model to return data matching a schema, and structured extraction is what you call it when that schema represents fields you want pulled out of a document rather than an action you want the model to take.

Do I need a fine-tuned model for structured extraction? Usually not. Modern general-purpose models handle structured extraction well out of the box when given a clear schema and tool use. Fine-tuning is worth considering only when you're running an extremely high volume of one narrow extraction type and want to shrink to a smaller, cheaper model without losing accuracy, not as a first step.

How do I handle PDFs and scanned documents before extraction? Run OCR or a document-parsing step first (many current models accept images or PDFs directly and handle layout reasonably well), then feed the resulting text or the document itself into your extraction call. Table-heavy and multi-column PDFs are the hardest case; test your OCR step's output quality on a sample before trusting downstream extraction accuracy numbers.

Can structured extraction guarantee 100% accuracy? No. Treat any LLM-based extraction pipeline as a probabilistic system with a known error rate you measure and monitor, not a deterministic parser. Design your downstream consumers (databases, workflows) to tolerate occasional bad records via validation, human review queues, and reconciliation checks rather than assuming perfect input.

Is structured extraction the same as retrieval-augmented generation (RAG)? No. RAG retrieves relevant context to help a model answer a question or generate text. Structured extraction pulls specific, typed fields out of a known document. They're often used together: RAG finds the right document or section, structured extraction pulls specific fields out of it.

What schema format should I use: JSON Schema, Pydantic, or something else? Use whatever your language's idiomatic validation library is (Pydantic in Python, Zod in TypeScript) and generate the JSON Schema from that, rather than hand-writing JSON Schema directly. This keeps your schema definition and your runtime validation in sync automatically instead of two artifacts that can drift apart.