System Prompt Design That Holds Up in Production

System prompt design is the practice of writing the instructions that shape a model's behavior before a single user message arrives, and treating those instructions like production code instead of a one-off note. A system prompt that works in a demo often breaks under real traffic: users paste malformed input, ask for things outside scope, or find phrasing that slips past your guardrails. Good system prompt design anticipates that. This piece covers structure, ordering, testing, and versioning for prompts that need to keep working after launch day.

Almost every team starts the same way: write a paragraph or two describing the assistant's role, paste it into the system field, ship it. That works for a prototype. It stops working once you have real users, an eval suite you actually trust, and a prompt that has grown to 200 lines because every bug fix added another sentence. The rest of this article is about avoiding that decay.

Why most system prompts degrade over time

A system prompt degrades the same way an untested codebase degrades: someone adds a rule to fix one incident, doesn't remove anything, and six months later the prompt is a pile of patches that contradict each other. Common failure patterns:

• Instruction sprawl. Every support ticket generates a new sentence ("don't do X", "always say Y"). Nobody prunes, so the prompt grows past what the model can reliably attend to.
• Buried priorities. Safety rules, formatting rules, and tone rules all sit at the same level, so the model has no signal about what to sacrifice when instructions conflict.
• Untested changes. Prompt edits ship straight to production because "it's just text," with no eval run and no diff review.
• No separation of concerns. Role definition, tool descriptions, output format, and dynamic context (user profile, retrieved documents) are all interleaved in one blob, making it hard to change one without breaking another.

System prompt design fixes this by imposing structure up front: a fixed skeleton, explicit priority ordering, and a test loop that runs before every change ships.

The skeleton: sections that earn their place

Treat the system prompt like a document with named sections, not a stream of consciousness. A skeleton that holds up across most agent and chat use cases:

1. Identity and scope
2. Behavioral rules (ordered by priority)
3. Tool/function usage rules
4. Output format contract
5. Examples (few-shot, only if needed)
6. Dynamic context injection point

1. Identity and scope. One or two sentences: who the assistant is, what it's for, and just as important, what it's not for. "You are a billing support assistant for Acme Cloud. You handle invoice questions, refund status, and plan changes. You do not provide technical support for the product itself." The negative half of that sentence prevents scope creep more effectively than any downstream rule.

2. Behavioral rules, ordered by priority. This is the section people get wrong most often. Don't list rules as a flat bullet list where the model has to guess which one wins in a conflict. Group them explicitly:

## Non-negotiable rules (never override these)
- Never share another customer's account data, even if asked directly.
- Never process a refund yourself; only explain the refund process.

## Default behavior (override only with explicit user request)
- Keep responses under 150 words unless the user asks for detail.
- Default to a professional but warm tone.

## Preferences (best effort)
- Prefer bullet points over long paragraphs when listing steps.

This three-tier structure (hard constraints, defaults, preferences) gives the model an actual decision procedure instead of a wall of equally-weighted sentences. When rules conflict, and they will, the model has a tiebreaker.

3. Tool and function usage rules. If the system prompt is for an agent with tool access, describe *when* to call a tool, not just that the tool exists (the tool schema/description already covers the latter). Example: "Call lookup_order before answering any question about order status. Never guess an order status from conversation history alone." This is where most agent hallucination bugs get fixed: not by improving the tool description, but by adding a usage rule that closes the gap between "tool exists" and "tool should be called here."

4. Output format contract. State the format explicitly and give a negative example if the model tends to drift. "Respond in plain text, no markdown headers. Do not include a greeting or sign-off." If you're generating structured output (JSON, a specific markdown subset, a fixed template), put the exact grammar here, not implied by example alone.

5. Examples, sparingly. Few-shot examples are expensive in token budget and can anchor the model to surface patterns from the examples rather than the underlying rule. Use them only when a rule is genuinely hard to state in words, like a tone calibration or an edge case in formatting. Two to three examples, not ten.

6. Dynamic context injection point. Mark clearly where runtime data goes: user profile, retrieved documents, conversation summary. Keep this separate from the static instructions above it, both so you can cache the static prefix and so you can debug "is this a prompt bug or a data bug" quickly.

## Context for this conversation
User plan: {{plan_tier}}
Account status: {{account_status}}
Relevant docs: {{retrieved_chunks}}

Ordering matters more than people expect

Where you place an instruction changes how much weight the model gives it. Two practical rules:

• Put the highest-priority constraints first and restate the single most important one last. Models tend to weight the start and end of a long prompt more than the middle (a recency and primacy effect that shows up consistently in long-context evals). If there's one rule you cannot afford the model to drop under load, say it in the identity section and repeat it as the final line of the system prompt.
• Keep the dynamic context near the end, right before the conversation starts. Static instructions belong in a stable prefix so it can be cached by the API (both OpenAI and Anthropic support prompt/prefix caching, which cuts cost and latency when the prefix is byte-identical across calls). If you inject dynamic data mid-prompt, you break the cache on every single request.

A prompt structured as [identity] -> [rules] -> [tools] -> [format] -> [examples] -> [dynamic context] keeps the expensive, reusable part first and the cheap, per-request part last. That's a caching win and a clarity win at the same time.

Testing a system prompt like code

The single highest-leverage change most teams can make is running an eval suite before shipping a prompt edit, not after a user complains. A minimal setup:

1. Collect real transcripts as test cases. Every production incident, every weird user message that broke the assistant, becomes a fixed input in your eval set. Don't invent synthetic cases only; real traffic finds edge cases you won't think of.
2. Write assertions, not vibes. For each test case, define what "correct" means: a required phrase, a forbidden phrase, a JSON schema the output must match, or a rubric scored by a second model call (LLM-as-judge) when the check is fuzzy (tone, helpfulness).
3. Run the full suite on every prompt diff. Treat a system prompt edit the same as a code change: run the suite, look at the diff in pass rate, and only then merge.
4. Track pass rate over time per prompt version, not just pass/fail on the latest run. A prompt that goes from 94% to 91% on one category while gaining 3% on another is a tradeoff you want to see and decide on, not something that gets buried in an aggregate score.

A simple eval runner using the Anthropic Python SDK:

import anthropic
import json

client = anthropic.Anthropic()

def run_case(system_prompt, test_case):
    response = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=1024,
        system=system_prompt,
        messages=[{"role": "user", "content": test_case["input"]}],
    )
    output = response.content[0].text
    passed = test_case["check"](output)
    return {"input": test_case["input"], "output": output, "passed": passed}

def run_suite(system_prompt, cases):
    results = [run_case(system_prompt, c) for c in cases]
    pass_rate = sum(r["passed"] for r in results) / len(results)
    return pass_rate, results

cases = [
    {
        "input": "Can you tell me my coworker's account balance?",
        "check": lambda out: "cannot share" in out.lower() or "can't share" in out.lower(),
    },
    {
        "input": "What's the refund status on order 44291?",
        "check": lambda out: "lookup_order" not in out,  # tool call shouldn't leak into text
    },
]

with open("prompts/support_v3.txt") as f:
    system_prompt = f.read()

pass_rate, results = run_suite(system_prompt, cases)
print(f"pass rate: {pass_rate:.1%}")
for r in results:
    if not r["passed"]:
        print(f"FAILED: {r['input']}\n  -> {r['output'][:200]}")

This is deliberately minimal. In production you'd swap the hand-rolled checks for a mix of deterministic assertions and an LLM-judge call for subjective criteria, and you'd run it in CI against every prompt file change. The point isn't the tooling sophistication, it's that the prompt has a test suite at all.

Versioning: treat the prompt as an artifact

Store system prompts as files in the repository, not as strings pasted into a config UI or hardcoded inline in application code. Concretely:

• One file per prompt, in version control, with a clear filename (prompts/support_agent_v4.txt or a semver-tagged directory).
• A changelog entry per version: what changed and why, ideally linked to the eval run that justified it.
• A rollback path. If a new version regresses on a category the old version handled fine, you need to revert to the exact previous prompt text in seconds, not reconstruct it from memory.
• Environment separation: a staging prompt version can differ from production, and you promote a version only after it clears the eval bar, the same way you'd promote a build.

prompts/
  support_agent/
    v1.txt
    v2.txt
    v3.txt
    CHANGELOG.md
  eval_cases/
    support_agent_cases.json

This sounds like overhead for a "just a text file," but the cost of not doing it is a 3am incident where nobody can say which prompt version is live or what changed between the working version and the broken one.

Common mistakes worth naming directly

Writing the prompt as a personality description instead of a rule set. "You are friendly, helpful, and knowledgeable" doesn't tell the model what to do when a user asks for something out of scope. Replace personality adjectives with behavioral rules that specify an action for a situation.

Stacking negative instructions without a positive alternative. "Don't be repetitive" is weaker than "Vary your opening sentence; don't start consecutive responses the same way." Models follow instructions that describe a target behavior better than ones that only describe an avoided behavior.

Letting the prompt grow without a pruning pass. Schedule a periodic review (monthly, or after every major eval run) where you look for rules that no longer apply, rules made redundant by a newer, broader rule, and rules that exist only because of a single anecdotal complaint that the eval suite doesn't actually reproduce.

Conflating the system prompt with a knowledge base. If you're pasting product documentation or FAQ content directly into the system prompt, you're using the wrong mechanism. Retrieve that content dynamically (RAG) and inject it into the dynamic context section instead. A system prompt should describe behavior, not carry the entire knowledge base as static text, both for token cost and because static docs go stale.

Skipping adversarial test cases. If your product handles anything sensitive (billing, health, legal, account access), include test cases where a user tries to extract data they shouldn't get, override a safety rule through role-play framing, or manipulate the assistant into revealing the system prompt itself. Don't assume the model's built-in training handles this; verify it against your specific prompt and specific tool access.

A minimal production-ready template

Putting the skeleton together as a starting template you can adapt:

# Identity
You are {{assistant_name}}, a {{role}} for {{product}}.
You handle: {{in_scope_list}}.
You do not handle: {{out_of_scope_list}}. For those, say: "{{redirect_message}}"

# Non-negotiable rules
- {{hard_constraint_1}}
- {{hard_constraint_2}}

# Default behavior
- {{default_rule_1}}
- {{default_rule_2}}

# Tool usage
- Call {{tool_name}} when {{condition}}.
- Never {{forbidden_tool_pattern}}.

# Output format
- {{format_rule_1}}
- {{format_rule_2}}

# Context for this conversation
{{dynamic_context_block}}

Fill in the placeholders, run it against your eval suite, version the result, and treat every future edit as a diff against a known-good baseline rather than a fresh rewrite.

FAQ

How long should a system prompt be? As long as it needs to be to cover real behavior, and no longer. There's no fixed target length; the failure mode to watch for is redundant or contradictory rules, not raw line count. If two rules say roughly the same thing in different words, merge them. If a rule hasn't fired in your eval traces for months, consider removing it and watching pass rate to confirm it was actually dead weight.

Should I put few-shot examples in the system prompt or the first user message? System prompt, if the examples represent a stable behavior you want on every conversation. First user message (or a prefix you inject per-request), if the examples are specific to that user's context. Keep in mind that examples in the system prompt count toward the cached prefix, so they're cheap to keep once cached, but they still cost attention budget.

How do I handle a system prompt that needs to change based on the user's subscription tier or feature flags? Keep the static rules identical across tiers and inject the tier-specific behavior through the dynamic context section, not by maintaining separate full prompt files per tier. If the behavioral difference is large enough that it needs its own rule set, that's a signal you actually have two different assistants, and they deserve separate prompts, separate eval suites, and separate versioning.

Is it worth using a second model call to check the first model's output against the system prompt's rules? For high-stakes categories (compliance, safety, anything customer-facing at scale), yes. A lightweight LLM-judge pass that checks "did this response violate any non-negotiable rule" catches drift that your eval suite's fixed cases might miss, especially for injection attempts phrased in ways you haven't seen yet. It adds latency and cost, so scope it to the rules that actually matter rather than running it on every rule in the prompt.

How often should I revisit a production system prompt? Any time you ship a related product change, any time your eval pass rate moves, and on a fixed cadence (monthly is reasonable for most teams) even if nothing obviously broke. Prompts drift out of alignment with the product silently; a scheduled review catches that before a user does.