Prompt EngineeringDSPyLLM OptimizationPythonClaude API

Prompt Optimization with DSPy

Pramod Dutta · Jul 1, 2026 · 15 min read

DSPy optimization is the practice of letting a program search for the best prompt for your task instead of you hand-writing and hand-tweaking it. You define a pipeline of typed steps (signatures and modules), write a metric that scores whether an output is good, and hand both to a DSPy optimizer, which runs your program against training examples and rewrites instructions and few-shot demonstrations until the metric improves. The result is a prompt (or set of prompts, if your pipeline has multiple steps) that was found by search, not by guesswork, and that you can re-optimize the moment you change models or data.

This matters because prompt engineering by hand doesn't scale. You tweak a system prompt, run it against ten examples, eyeball the outputs, tweak again. That loop breaks down once you have a multi-step pipeline, a metric that isn't "did it look right to me," or a need to swap models without starting over. DSPy treats prompts as parameters of a program and treats prompt optimization as a compilation step, with the same discipline you'd apply to hyperparameter search in a normal ML project.

What DSPy Optimization Actually Does

Traditional prompt engineering couples two things that don't need to be coupled: the logic of your task (what steps happen, in what order, with what inputs and outputs) and the wording that gets a specific model to execute that logic well. DSPy separates them.

You write:

Signatures: a typed description of an input/output transformation, like "question -> answer" or "document, question -> answer, confidence".
Modules: a strategy for how the LM should be prompted to fulfill a signature, such as direct prediction (dspy.Predict) or step-by-step reasoning (dspy.ChainOfThought).
Programs: modules composed together, with regular Python control flow between them.

None of this contains prompt wording. The wording, few-shot examples, and (with newer optimizers) even instruction text are filled in by an optimizer (DSPy calls these "teleprompters" internally, though the current public API just calls them optimizers). You give the optimizer:

A program.
A metric function that takes an example and a prediction and returns a score.
A training set of examples (inputs, and ideally expected outputs or at least enough signal to score against).

The optimizer then runs a search: it tries different instructions, different selections of few-shot demonstrations pulled from your training set, sometimes different combinations of sub-prompts across a multi-step pipeline, and keeps whatever scores highest on a held-out validation split. What you get back is a compiled program: the same Python logic, but with concrete instructions and examples baked into each module's prompt.

This reframes "prompt optimization" as a search-and-evaluate loop rather than an art. It's the same idea as tuning a scikit-learn pipeline with GridSearchCV, except the parameters being searched are text (instructions, demonstration sets) instead of numbers.

Setting Up DSPy with an LLM Backend

Install DSPy and configure a language model. DSPy talks to models through a dspy.LM wrapper that uses LiteLLM-style provider prefixes under the hood, so switching providers is a one-line change.

pip install -U dspy

Configure Claude as the backend:

import dspy

# Provider-prefixed model string; check your provider's current
# model catalog for the exact ID you want to target.
lm = dspy.LM(
    "anthropic/claude-opus-4-8",
    api_key="YOUR_ANTHROPIC_API_KEY",
    max_tokens=4096,
)
dspy.configure(lm=lm)

A few notes on this setup:

The anthropic/<model-id> prefix tells DSPy's LiteLLM layer to route through the Anthropic API. Model IDs change over time as new versions ship, so treat the string above as an example, not a hardcoded constant. Pull the current model ID from Anthropic's model documentation before you deploy, and keep it in an environment variable or config file rather than inline in code, so upgrading the model later doesn't mean hunting through your codebase.
You can point dspy.LM at any Claude-compatible endpoint (first-party API, or a cloud provider's hosted Claude) by changing the model string and credentials, without touching your signatures, modules, or optimizer configuration. That's the main structural benefit of keeping DSPy's model config separate from your program: re-running optimization against a different or newer model is a config change, not a rewrite.
If you're optimizing against a reasoning-capable model, DSPy supports the model's reasoning/thinking output as part of its handling of chat models, but you don't need to configure anything special for a first pass. Start simple, verify the pipeline works end-to-end, then tune.
For teams optimizing at scale, it's common to use a cheaper/faster model during the search phase (to keep optimization cost and latency down) and a stronger model for final production serving. DSPy supports this directly: pass one dspy.LM for prompt_model and another for task_model in optimizers that expose the distinction, or just re-run compilation once against your target model when you're ready to lock it in.

Defining Signatures and Modules

Signatures describe the shape of a task. The shorthand string form is fine for simple cases:

classify = dspy.Predict("review_text -> sentiment: str, confidence: float")

result = classify(review_text="The shipping was slow but the product itself is great.")
print(result.sentiment, result.confidence)

For anything you plan to optimize seriously, prefer the class-based form. It gives the optimizer more surface area to work with (field descriptions become part of what gets tuned) and makes your intent explicit:

class ClassifySupportTicket(dspy.Signature):
    """Classify a customer support ticket by urgency and category."""

    ticket_text: str = dspy.InputField(desc="Raw text of the support ticket")
    urgency: str = dspy.OutputField(desc="One of: low, medium, high, critical")
    category: str = dspy.OutputField(desc="One of: billing, bug, feature_request, account, other")
    reasoning: str = dspy.OutputField(desc="One-sentence justification for the classification")


classify = dspy.ChainOfThought(ClassifySupportTicket)

dspy.ChainOfThought wraps the signature so the model reasons before producing the output fields, which usually helps on tasks where the label depends on synthesizing multiple details from the input. dspy.Predict skips the reasoning step and is faster/cheaper when the task is simple enough that reasoning doesn't add accuracy.

Multi-step programs compose modules with plain Python:

class TicketTriage(dspy.Module):
    def __init__(self):
        super().__init__()
        self.classify = dspy.ChainOfThought(ClassifySupportTicket)
        self.draft_reply = dspy.ChainOfThought("ticket_text, category, urgency -> draft_reply")

    def forward(self, ticket_text: str):
        classification = self.classify(ticket_text=ticket_text)
        reply = self.draft_reply(
            ticket_text=ticket_text,
            category=classification.category,
            urgency=classification.urgency,
        )
        return dspy.Prediction(
            urgency=classification.urgency,
            category=classification.category,
            draft_reply=reply.draft_reply,
        )


triage = TicketTriage()

This is a normal Python class. forward can branch, loop, call external tools, or call other DSPy modules. The optimizer's job later is to find good instructions and demonstrations for self.classify and self.draft_reply independently, given how they're actually used inside forward.

Building a Metric for DSPy Optimization

The metric is the single most important piece of a DSPy optimization run. It's the function the optimizer is searching to maximize, so a sloppy metric produces a prompt that's good at satisfying the sloppy metric, not good at your actual task.

A metric is a function (example, prediction, trace=None) -> score, where example is a labeled item from your dataset and prediction is what your program produced for that item's inputs.

Simple exact-match metric:

def category_match(example, prediction, trace=None):
    return example.category == prediction.category

Metrics don't have to be binary. For the ticket triage example, you might want partial credit for getting urgency right even if category is wrong, or vice versa:

def triage_metric(example, prediction, trace=None):
    score = 0.0
    if prediction.category == example.category:
        score += 0.5
    if prediction.urgency == example.urgency:
        score += 0.5
    return score

For tasks without a clean gold label, LLM-as-judge metrics are common: call a model to grade the prediction against a rubric and return a numeric score. Keep judge prompts narrow and specific ("does the reply acknowledge the customer's stated problem, yes or no") rather than open-ended ("is this a good reply"), because the judge's own reliability puts a ceiling on how much signal your optimizer gets.

judge = dspy.Predict("ticket_text, draft_reply -> acknowledges_problem: bool, is_polite: bool")

def reply_quality_metric(example, prediction, trace=None):
    verdict = judge(ticket_text=example.ticket_text, draft_reply=prediction.draft_reply)
    return float(verdict.acknowledges_problem) * 0.6 + float(verdict.is_polite) * 0.4

Two rules that save real debugging time later:

Keep the metric deterministic where possible. If it calls an LLM judge, the search itself becomes noisier, which means you need a larger validation set to trust the results and you should expect run-to-run variance.
Trace-aware metrics unlock stricter optimizers. Some optimizers pass a non-None trace argument during the search phase (but None during final evaluation), which lets you write a metric that behaves as "all-or-nothing" during search (return True/False for whether the full pipeline's intermediate steps were correct) but returns a continuous score during evaluation. Check the metric signature examples in the optimizer you're using before assuming your metric is being called the same way in both phases.

Building a Training Set

DSPy optimizers need example data structured as dspy.Example objects, with .with_inputs() marking which fields are inputs versus expected outputs:

trainset = [
    dspy.Example(
        ticket_text="My card was charged twice for the same order, please refund the duplicate.",
        category="billing",
        urgency="medium",
    ).with_inputs("ticket_text"),
    dspy.Example(
        ticket_text="The app crashes immediately on launch, I can't use it at all.",
        category="bug",
        urgency="critical",
    ).with_inputs("ticket_text"),
    # ... more examples
]

You don't need thousands of examples. DSPy optimizers are designed to work with small training sets, often 20 to 100 labeled examples, because the search space is instructions and demonstration selection, not model weights. What matters more than volume is that the examples are representative of the range of inputs you expect in production, including edge cases and the categories your metric weights most heavily.

Split into train and validation sets. The optimizer uses the training set to construct candidate prompts (pulling few-shot examples from it) and the validation set to score those candidates and pick a winner. Reusing the same examples for both leaks information and inflates your apparent score.

import random

random.seed(0)
random.shuffle(trainset)
split = int(len(trainset) * 0.7)
train, val = trainset[:split], trainset[split:]

Running DSPy Optimization

DSPy ships several optimizers that trade off search cost against how much of the prompt they're allowed to change. Start cheap, escalate only if the result isn't good enough.

BootstrapFewShot : the baseline optimizer. It runs your program on training examples, keeps the ones where your metric scores well, and uses those as few-shot demonstrations inside each module's prompt. It doesn't rewrite instructions, only selects demonstrations.

from dspy.teleprompt import BootstrapFewShot

optimizer = BootstrapFewShot(metric=triage_metric, max_bootstrapped_demos=4, max_labeled_demos=4)
compiled_triage = optimizer.compile(triage, trainset=train)

BootstrapFewShotWithRandomSearch : the same idea, but it tries multiple random selections of demonstrations and multiple bootstrapping runs, then keeps whichever combination scores best on validation. More expensive, generally better than plain BootstrapFewShot.

from dspy.teleprompt import BootstrapFewShotWithRandomSearch

optimizer = BootstrapFewShotWithRandomSearch(
    metric=triage_metric,
    max_bootstrapped_demos=4,
    num_candidate_programs=8,
)
compiled_triage = optimizer.compile(triage, trainset=train, valset=val)

MIPROv2 : the current strong general-purpose optimizer for most tasks. It optimizes both instructions and few-shot demonstrations jointly, using a separate model to propose candidate instructions grounded in your program's behavior and data, then runs Bayesian optimization over the combined search space of instructions and demonstration sets.

from dspy.teleprompt import MIPROv2

optimizer = MIPROv2(
    metric=triage_metric,
    auto="medium",  # "light" | "medium" | "heavy" controls search budget
)
compiled_triage = optimizer.compile(
    triage,
    trainset=train,
    valset=val,
    requires_permission_to_run=False,
)

The auto parameter is the main knob: "light" runs a fast, cheap search suitable for iterating on your pipeline; "heavy" spends far more compute exploring the instruction and demonstration space and is what you'd run once you've validated the pipeline shape and are optimizing for a production deployment.

Once compiled, save the result so you don't have to re-run optimization every time you restart your process:

compiled_triage.save("triage_compiled.json")

# Later, in a fresh process:
triage = TicketTriage()
triage.load("triage_compiled.json")

The saved artifact contains the optimized instructions and demonstration sets DSPy found, tied to the program structure you compiled. Load it back into an identically structured program to use it in production without paying the optimization cost again.

Evaluating Before and After

Optimization without a before/after comparison is just vibes with extra steps. Use dspy.Evaluate to score the uncompiled and compiled programs on the same held-out set:

from dspy.evaluate import Evaluate

evaluator = Evaluate(devset=val, metric=triage_metric, num_threads=8, display_progress=True)

baseline_score = evaluator(triage)
optimized_score = evaluator(compiled_triage)

print(f"Baseline: {baseline_score}")
print(f"Optimized: {optimized_score}")

If the optimized score isn't meaningfully better than baseline, don't assume the optimizer failed silently and move on. Check, in order: whether your metric actually differentiates good from bad outputs (run it manually against a few known-good and known-bad predictions), whether your training set covers the failure modes you care about, and whether the signature's field descriptions give the model enough context to succeed even with a good prompt. DSPy optimization can't fix a task that's fundamentally underspecified at the signature level.

It's also worth testing the compiled program against inputs that weren't in either your train or validation set, since a small validation set can still be exhausted by search-time overfitting on tasks with high metric variance.

Common Pitfalls

Optimizing on too few examples. A validation set of five examples produces a score with huge variance; the optimizer might "win" by getting lucky on those five rather than by actually generalizing. Aim for at least 20 to 30 validation examples before trusting a score difference.

A metric that's easier to satisfy than the real task. If your metric is "does the output contain the word 'urgent' when urgency is high," the optimizer will happily learn to stuff that word in regardless of whether the reasoning behind it is sound. Write metrics against the actual downstream requirement, not a proxy that's convenient to compute.

Re-optimizing on every model swap without checking cost. MIPROv2 at "heavy" search settings makes many calls to both the model being optimized and (if used) the model proposing instructions. Budget for this, especially if you're iterating on signatures and re-running the full search each time. Use "light" while you're still shaping the pipeline, escalate once the shape is stable.

Treating the compiled prompt as final. A compiled program is optimized against the training distribution and metric you gave it at that moment. If your production traffic drifts, or your task requirements change, the old compiled prompt degrades the same way any static, hand-written prompt would. Re-run optimization on fresh data periodically, the same way you'd retrain a model on fresh data.

Skipping the "does the raw signature even work" check. Before spending optimizer budget, run the uncompiled program manually against a handful of examples with dspy.inspect_history(n=1) to see the actual prompt DSPy sent and the raw model response. If the base signature is confused about what it's supposed to output, no amount of few-shot search fixes that; fix the signature and field descriptions first.

Forgetting to pin the model version in production. Because DSPy's compiled artifacts are tied to the prompt and demonstrations, not to model weights, a compiled program stays valid across model upgrades in the sense that it will still run. But its quality was measured against a specific model. If you swap claude-opus-4-8 for a newer model ID without re-running evaluation, you're deploying on faith. Re-evaluate (and ideally re-optimize) after any model change.

FAQ

Does DSPy optimization replace prompt engineering entirely? No. You still write signatures, choose module types (Predict vs ChainOfThought vs others), and design the metric, all of which require the same domain understanding good prompt engineering always required. What DSPy replaces is the manual trial-and-error loop of wording instructions and picking few-shot examples by hand. Think of it as automating the mechanical part of prompt tuning while you own the task design.

How much does running an optimizer like MIPROv2 cost? It depends on your training set size, the auto setting, and the models involved. "light" search on a small dataset (20 to 40 examples) is comparable to running your evaluation set through the model a handful of times. "heavy" search can run into hundreds of model calls, since it's exploring many candidate instruction/demonstration combinations. Start with "light" to validate your pipeline and metric are sound, then escalate.

Can I use DSPy optimization with a multi-step agent, not just a single classification task? Yes. Compose dspy.Module subclasses the same way you'd compose any Python classes, with each step being its own signature and module. The optimizer treats each module's prompt as a separate parameter to search, using the metric evaluated on the full pipeline's final output (or on intermediate steps too, if your metric inspects the trace). This is where DSPy's value is highest, because manually hand-tuning prompts across several interacting steps is exactly the kind of combinatorial problem search handles better than intuition.

Do I need labeled ground-truth data, or can I optimize with just a metric? You need at least inputs; labels help but aren't strictly required if your metric can score outputs without them, such as an LLM-judge metric that grades against a rubric rather than an exact answer. Optimizers that rely purely on bootstrapping few-shot examples do need some way to decide which generated outputs are "good enough" to keep as demonstrations, which is where the metric does the work labels would otherwise do.

How do I know if I should use BootstrapFewShot or MIPROv2? Start with BootstrapFewShot or BootstrapFewShotWithRandomSearch when you want a fast baseline or your task is simple enough that good few-shot examples alone move the metric. Move to MIPROv2 when demonstration selection alone plateaus and you suspect the instructions themselves need rewriting, or when you're optimizing a multi-step pipeline where instruction wording at each step interacts with the others in ways that are hard to reason about by hand.

Will a DSPy-optimized prompt work if I change the underlying model later? The compiled program will still run, but its quality was validated against whatever model you optimized with. Different models respond differently to the same instructions and few-shot examples, so treat a model swap as a trigger to re-run your evaluation set, and re-optimize if the score drops. Keeping the model ID in a config variable rather than hardcoded makes this swap-and-re-evaluate cycle a one-line change instead of a rewrite.