Few-Shot vs Zero-Shot Prompting: When Examples Help

Few shot vs zero shot is not a style preference, it is a debugging decision. If your model's output format is inconsistent, its tone drifts, or it misreads ambiguous inputs, adding two or three examples to the prompt usually fixes it faster than rewriting your instructions. If the model already does the task correctly with a plain instruction, adding examples just burns tokens and latency for no gain. This article walks through how to tell which situation you're in, how to build few-shot examples that actually generalize, and where the technique breaks down.

What zero-shot and few-shot actually mean

Zero-shot prompting means you describe the task in natural language and give the model no worked examples. You state the instruction, maybe some constraints, and let the model produce output based on what it learned during training.

Classify the sentiment of this review as positive, negative, or neutral.

Review: "The battery life is decent but the screen scratches easily."
Sentiment:

Few-shot prompting means you show the model a handful of input-output pairs before asking it to do the same thing on a new input. The examples live in the same prompt, in the same context window, no fine-tuning involved.

Classify the sentiment of each review as positive, negative, or neutral.

Review: "Fast shipping and the case fits perfectly."
Sentiment: positive

Review: "Arrived broken, packaging was fine but the unit doesn't power on."
Sentiment: negative

Review: "It's fine. Does what it says, nothing special."
Sentiment: neutral

Review: "The battery life is decent but the screen scratches easily."
Sentiment:

Both are forms of in-context learning: the model adapts its behavior using only what's in the prompt, not through weight updates. The difference is entirely about how much demonstration you give it before the real input.

There's a middle category worth naming: one-shot, a single example. In practice, one-shot behaves like a weak version of few-shot, it helps with format but rarely fixes reasoning gaps the way three-plus examples can. Most of what applies to few-shot below applies at n=1 too, just less reliably.

Why zero-shot is the right default

Start every prompt zero-shot. Not because it's simpler to write (though it is), but because it's the fastest way to find out what the model actually needs.

Three concrete reasons to default to zero-shot:

Token cost and latency. Every example you add to a prompt gets re-processed on every single call. If you're classifying ten thousand support tickets a day and your few-shot block adds 400 tokens, that's 4 million extra input tokens daily for demonstration text that produces zero incremental value once the model already understands the task.

Modern instruction-tuned models are good at zero-shot. Models trained with reinforcement learning from human feedback and extensive instruction tuning follow direct instructions far better than raw base models did. For well-specified, common tasks like summarization, translation, straightforward classification, and simple extraction, zero-shot with a clear instruction plus a couple of output constraints is often enough.

Examples can leak bias you didn't intend. If your three sentiment examples happen to all be about electronics, the model may start weighting electronics-specific vocabulary more heavily than it should for reviews about, say, furniture. Every example is also an implicit statement about the distribution of inputs you expect, and a poorly chosen set narrows the model's behavior in ways you didn't design for.

The workflow that works well in practice: write the zero-shot version, run it against 20-30 real examples pulled from your actual data (not ones you invented), and look at the failures. If the model fails because it doesn't understand the task, that's an instruction problem, rewrite the instruction. If it fails because it doesn't understand your output format or gets confused by a specific edge case, that's usually a few-shot problem.

When few-shot earns its token cost

Few-shot prompting solves problems that clearer instructions cannot. Here are the situations where it consistently helps.

Format enforcement. Telling a model "output valid JSON with keys name, category, price" is a good start, but models still occasionally add commentary, use inconsistent key casing, or wrap output in markdown code fences you didn't ask for. Showing two or three exact input-output pairs in your target format is far more reliable than describing the format in prose, because the model is literally pattern-matching against what it just saw rather than interpreting a specification.

Extract product info as JSON with keys name, category, price_usd.

Input: "The UltraBrew 3000 coffee maker is on sale for $89.99, filed under kitchen appliances."
Output: {"name": "UltraBrew 3000", "category": "kitchen appliances", "price_usd": 89.99}

Input: "Nomad hiking boots, outdoor gear, currently $124.50."
Output: {"name": "Nomad hiking boots", "category": "outdoor gear", "price_usd": 124.50}

Input: "Wireless earbuds Pulse X2 are $59 in the audio section."
Output:

Domain-specific classification with fuzzy boundaries. If your categories aren't obvious from their names (internal ticket priority tiers, custom taxonomy labels, a support routing scheme specific to your product), examples convey the boundary better than a definition does. "Urgent" means something different at every company. Show the model three tickets you'd call urgent and two you wouldn't, and it calibrates fast.

Tone and style matching. If you need output in a specific voice (a brand's customer service tone, a terse internal-tool style, a particular technical writing register), examples transfer style far more efficiently than adjectives do. "Write in a friendly but professional tone" is vague; three example replies in that tone are precise.

Correcting a specific, recurring failure mode. If you've already tried zero-shot and found a consistent error pattern, such as the model always miscategorizing a certain kind of input, add one example that directly targets that failure. This is more efficient than a general few-shot set: you're patching a known hole, not demonstrating the whole task from scratch.

Multi-step or unusual output structures. Tasks that require the model to produce a structured chain (e.g., extract fields, then compute a derived value, then justify it in one sentence) benefit from seeing the full expected structure once or twice, since describing structure in prose gets clunky fast.

Building few-shot examples that generalize

Bad few-shot examples make output worse, not better. The failure mode is real: three near-identical examples teach the model a narrow pattern instead of the underlying task.

Cover the input space, not just the easy cases. If you're building a sentiment classifier, don't use three obviously positive and obviously negative reviews. Include a neutral one, a sarcastic one, a mixed one (good product, bad shipping). The examples should span the actual difficulty distribution of your real inputs, or the model will handle only the easy slice well.

Keep the format rigidly consistent across every example. If example one has a trailing period after the label and example two doesn't, the model will inherit that inconsistency. Every example should look like it came from the same automated pipeline, because that consistency is exactly what you're asking the model to reproduce.

Order matters more than people expect. Models show some sensitivity to example order, especially with smaller context windows or when examples share a category. If you have five examples and four are "negative," putting all four negatives first can bias the model toward over-predicting negative on ambiguous new inputs. Shuffle or deliberately alternate categories.

Fewer, well-chosen examples beat many mediocre ones. Three to five carefully selected examples, each demonstrating a distinct aspect of the task (a different edge case, a different format wrinkle), outperform ten examples that are all variations on the same easy case. Past a certain point, more examples add token cost without adding new information for the model to condition on.

Pull examples from real data, review them for correctness. Invented examples tend to be cleaner than reality and can teach the model a version of the task that's easier than the one it will actually face. Sample from your logs, verify the "correct" label by hand, then use those.

Here's a small runnable comparison using the Claude API's Python SDK, testing the same classification task zero-shot and few-shot so you can see the failure mode directly:

import anthropic

client = anthropic.Anthropic()

zero_shot_prompt = """Classify the urgency of this support ticket as low, medium, or high.

Ticket: "Hey, just wondering if you have a dark mode planned for next quarter?"
Urgency:"""

few_shot_prompt = """Classify the urgency of this support ticket as low, medium, or high.

Ticket: "Any plans to add keyboard shortcuts? Not urgent, just curious."
Urgency: low

Ticket: "Getting a 500 error on checkout for about 10% of users since this morning."
Urgency: high

Ticket: "Export to CSV is missing a column we used to have, workaround exists but annoying."
Urgency: medium

Ticket: "Hey, just wondering if you have a dark mode planned for next quarter?"
Urgency:"""

for label, prompt in [("zero-shot", zero_shot_prompt), ("few-shot", few_shot_prompt)]:
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=10,
        messages=[{"role": "user", "content": prompt}],
    )
    print(label, "->", response.content[0].text.strip())

Run this against a batch of your own tickets rather than one example, and track disagreement rate between zero-shot and few-shot outputs. A high disagreement rate on ambiguous tickets tells you the examples are doing real calibration work, not just cosmetic formatting.

Measuring whether few-shot is actually helping

Don't guess, test. Build a small labeled evaluation set (50-100 examples is enough to start) with ground truth you trust, and run both prompt versions against it.

A simple evaluation loop:

def evaluate(prompt_template, test_cases, client, model="claude-sonnet-4-5"):
    correct = 0
    for case in test_cases:
        full_prompt = prompt_template + case["input"]
        response = client.messages.create(
            model=model,
            max_tokens=10,
            messages=[{"role": "user", "content": full_prompt}],
        )
        prediction = response.content[0].text.strip().lower()
        if prediction == case["label"].lower():
            correct += 1
    return correct / len(test_cases)

Compare accuracy, but also compare format-compliance rate separately (did the output parse as valid JSON, did it match one of your allowed labels exactly) since few-shot often improves compliance without moving accuracy much, and that distinction changes what fix you should apply.

If few-shot doesn't beat zero-shot on your eval set, don't add it. This happens more often than prompt-engineering folklore suggests, especially on tasks the model already handles well from training. Paying the token tax for examples that don't move the needle is a real cost at production scale.

Common mistakes

Treating few-shot as a fix for a vague instruction. If your instruction is ambiguous, examples paper over the ambiguity for the cases you happened to demonstrate, but new edge cases will still confuse the model. Fix the instruction first, add examples second.

Using too many examples and hitting diminishing returns. Beyond roughly five to eight well-chosen examples, additional examples mostly add cost. If you need ten-plus examples to get acceptable behavior, that's usually a sign the task needs a different approach entirely, such as retrieval-augmented context, a rules-based pre-filter, or fine-tuning if volume justifies it.

Not versioning your examples. Few-shot examples are part of your prompt and should live in source control with the same review process as code. Treat a change to your example set as a change worth testing against your eval set before shipping.

Forgetting examples consume context budget. In long-running agent workflows where the prompt already carries tool definitions, conversation history, and retrieved documents, a bulky few-shot block competes for context space with everything else. Keep examples terse: trim to the minimum tokens that convey the pattern.

Assuming few-shot examples transfer across models. An example set tuned against one model's quirks doesn't necessarily transfer cleanly to a different model or a different version of the same model family. Re-run your eval set whenever you swap models.

FAQ

Does few-shot prompting always improve accuracy? No. For tasks the model already performs well zero-shot, few-shot examples often leave accuracy unchanged while adding token cost. Few-shot's biggest wins are in output format consistency and disambiguating fuzzy category boundaries, not in raw task accuracy for well-specified problems.

How many examples should I use? Start with three to five that cover distinct, representative cases from your real input distribution. Test whether adding a sixth or seventh changes your eval score meaningfully; if it doesn't, stop there. There's no fixed "correct" number, it depends entirely on how much the task's difficulty varies across your input space.

Can I mix zero-shot instructions with few-shot examples? Yes, and you generally should. A clear natural-language instruction followed by a small set of examples usually outperforms either alone: the instruction sets the general rule, the examples pin down format and edge cases the instruction didn't cover.

Is few-shot the same as fine-tuning? No. Few-shot examples live entirely in the prompt and are re-sent on every call, they don't change the model's weights. Fine-tuning updates the model itself using a training dataset and persists across calls without needing examples in every prompt. Few-shot is cheaper to iterate on; fine-tuning is worth it only at high volume where the per-call token savings outweigh the training and maintenance cost.

Does example order inside the prompt matter? Yes, to a measurable degree. Models can show mild recency and category-clustering bias, so alternate between categories rather than grouping all examples of one label together, and consider testing a shuffled ordering against your eval set if you notice skewed predictions on ambiguous inputs.

What's the difference between few-shot prompting and chain-of-thought prompting? They solve different problems and can be combined. Few-shot shows input-output pairs to demonstrate a pattern or format. Chain-of-thought shows or requests intermediate reasoning steps to improve accuracy on multi-step problems. You can write few-shot examples that include reasoning steps in each demonstration, which combines both techniques for tasks that need both format control and better reasoning.

Should I use few-shot examples for simple extraction tasks like pulling a date from text? Usually not needed. Simple, well-defined extraction tasks are exactly the case where zero-shot with a precise instruction and an explicit output format (e.g., "respond only with the date in YYYY-MM-DD format, no other text") performs well. Save few-shot budget for tasks with genuine ambiguity or format drift.