Decomposing Complex Tasks into Prompt Steps

Task decomposition prompting is the practice of breaking a single large request into a sequence of smaller prompts, each with one job, one expected output shape, and a clear handoff to the next step. When you ask a model to "research this market, write a report, and check the numbers" in one shot, it often skips the checking, blends research with writing, and produces an answer that looks confident but is wrong in ways you can't easily find. Split that same request into four steps (gather, structure, draft, verify) and you get an output where each stage can be inspected, retried, and swapped out independently. This article walks through when to decompose, how to design the chain, how to pass state between steps, and how to handle failures without rebuilding the whole pipeline.

Why Task Decomposition Prompting Beats the Mega-Prompt

A single long prompt asks the model to hold too many constraints in its head at once: tone, structure, factual accuracy, formatting, and edge cases, all resolved in one forward pass. Models are good at local coherence, they are much weaker at globally satisfying ten unrelated instructions simultaneously. The failure mode is predictable: early instructions in the prompt get followed, later ones get quietly dropped, and the parts that require the most care (fact-checking, arithmetic, cross-referencing) are the ones that slip first because they're the most expensive to actually do.

Decomposition fixes this by giving each step a narrow job description. A step that only extracts structured data from raw text has nothing else competing for attention. A step that only writes prose from already-structured data doesn't have to also verify facts. A step that only verifies doesn't have to also write. Each of these narrow jobs is something the model can do reliably, and reliability compounds: four 95%-reliable steps chained with a validation gate between them beat one 60%-reliable mega-prompt almost every time, because you catch the failures instead of shipping them.

There's a second, more practical reason this matters: debuggability. When a mega-prompt output is wrong, you don't know if the model misunderstood the task, hallucinated a fact, or got the format wrong, because everything happened in one opaque generation. When a decomposed chain fails, the broken step is visible. You look at the intermediate output, see the extraction step returned malformed JSON, and you know exactly what to fix without touching anything else.

When to Decompose (and When Not To)

Decomposition adds latency and orchestration code, so it isn't free. Use it when at least one of these is true:

• The task has genuinely separable sub-goals (research, then write, then cite)
• Any sub-goal requires a different "mode" of reasoning (extraction versus creative writing versus arithmetic)
• You need to validate or re-run one part without regenerating everything
• The output format for one stage doesn't match the input format the model naturally produces for the next
• The task is long enough that quality visibly degrades partway through a single response

Skip decomposition when the task is genuinely a single cohesive act, like "rewrite this paragraph to be more concise" or "classify this ticket into one of five categories." Splitting a one-step task into three steps just adds latency and three more places for something to go wrong. A good rule of thumb: if you can't describe what each sub-step outputs in one sentence, the task probably isn't ready to be split, and you should sketch the pipeline on paper before writing any code.

The Core Pattern: Plan, Execute, Verify

Most useful decompositions collapse into a three-stage shape, even when the middle stage has multiple sub-steps of its own.

Plan. A short step that turns the vague user request into a concrete, ordered list of sub-tasks. This step doesn't do any of the real work, it just decides what work needs doing and in what order. Keeping planning separate means you can log the plan, show it to a user for approval, or swap in a different plan without touching the execution logic.

Execute. One prompt per sub-task from the plan, run in sequence (or in parallel where sub-tasks don't depend on each other). Each execution step gets exactly the context it needs, nothing more, which keeps its prompt short and its failure surface small.

Verify. A final pass that checks the combined output against the original request: did we answer the question, are the numbers internally consistent, is anything missing. This step is cheap compared to the generation steps and catches the class of error where every individual step succeeded but the combination doesn't actually satisfy the user.

Here's the skeleton in code. The call_llm function is a stand-in for whatever client you use; swap in your provider's SDK.

def call_llm(system, user, model="claude-sonnet-4-5"):
    # Replace with your actual client call, e.g. anthropic.Anthropic().messages.create(...)
    response = client.messages.create(
        model=model,
        max_tokens=1500,
        system=system,
        messages=[{"role": "user", "content": user}],
    )
    return response.content[0].text


def plan_step(request):
    system = (
        "Break the user's request into an ordered list of concrete sub-tasks. "
        "Return only a JSON array of short task descriptions, nothing else."
    )
    return call_llm(system, request)


def execute_step(task, context):
    system = (
        "You complete exactly one task using the provided context. "
        "Do not add commentary, do not attempt other tasks."
    )
    user = f"Task: {task}\n\nContext:\n{context}"
    return call_llm(system, user)


def verify_step(original_request, combined_output):
    system = (
        "Check whether the combined output fully satisfies the original request. "
        "Return JSON: {\"pass\": true|false, \"issues\": [list of strings]}."
    )
    user = f"Request: {original_request}\n\nOutput:\n{combined_output}"
    return call_llm(system, user)

Each function does one thing and returns a predictable shape. That predictability is what lets you write orchestration code around them without special-casing every possible model quirk.

Building a Multi-Step Prompt Chain

Once you have a plan, the orchestration loop is straightforward: iterate over sub-tasks, run each one, and accumulate results into a growing context that later steps can reference.

import json

def run_chain(request):
    raw_plan = plan_step(request)
    tasks = json.loads(raw_plan)

    results = []
    running_context = ""

    for task in tasks:
        output = execute_step(task, running_context)
        results.append({"task": task, "output": output})
        running_context += f"\n\n[{task}]\n{output}"

    combined = "\n\n".join(r["output"] for r in results)
    verdict = json.loads(verify_step(request, combined))

    return {
        "tasks": tasks,
        "results": results,
        "combined": combined,
        "verdict": verdict,
    }

Two details matter here. First, running_context grows with every step, so later sub-tasks can reference earlier outputs ("using the pricing tiers from the previous step, write the comparison section"). Second, the verify step runs against the fully combined output, not against each piece in isolation, because internal consistency is exactly the thing that individual steps can't check about themselves.

For sub-tasks that don't depend on each other, run them concurrently instead of in a loop. If your plan step tags each task with its dependencies, you can group independent tasks into a batch and fire them off together, which cuts wall-clock latency substantially on a five- or six-step chain.

import concurrent.futures

def run_independent_batch(tasks, context):
    with concurrent.futures.ThreadPoolExecutor(max_workers=len(tasks)) as pool:
        futures = {pool.submit(execute_step, t, context): t for t in tasks}
        return {futures[f]: f.result() for f in concurrent.futures.as_completed(futures)}

Passing State Between Steps

The single most common bug in decomposed pipelines is losing information between steps because it wasn't put into a format the next step can actually parse. Free text is easy for a model to write but brittle for the next step to consume. Wherever a step's output feeds directly into another prompt or into your own code, force structured output.

def extract_step(raw_text):
    system = (
        "Extract the following fields as JSON: company_name, revenue_figure, "
        "revenue_period, source_quote. If a field is not present, use null."
    )
    return json.loads(call_llm(system, raw_text))

Two practical habits keep this reliable:

• Always specify a schema in the prompt itself, listing every field name and type, so the model isn't guessing what shape you want.
• Parse defensively. Wrap json.loads in a try/except and, on failure, re-prompt the same step once with the parse error appended ("Your last response was not valid JSON: {error}. Return only valid JSON matching the schema.") before falling back to a manual review queue.

def extract_with_retry(raw_text, max_attempts=2):
    system = (
        "Extract the following fields as JSON: company_name, revenue_figure, "
        "revenue_period, source_quote. If a field is not present, use null."
    )
    user = raw_text
    for attempt in range(max_attempts):
        raw = call_llm(system, user)
        try:
            return json.loads(raw)
        except json.JSONDecodeError as e:
            user = f"{raw_text}\n\nYour previous output was invalid JSON ({e}). Return only valid JSON."
    raise ValueError("extract_step failed after retries")

This one pattern, structured intermediate output plus a bounded retry, eliminates most of the flakiness people blame on "the model being unreliable." The model is usually fine; the handoff format was the weak point.

Handling Failures Mid-Chain

A chain of five steps has five places to fail, so plan for partial failure from the start rather than treating it as an edge case.

Fail fast on structural errors, retry on content errors. If a step returns malformed JSON, retry immediately as shown above. If a step returns well-formed but low-quality content (verify_step flags it), don't blindly retry the same prompt, because you'll likely get the same mediocre result. Instead, feed the verify step's specific complaint back into the execute step as extra context.

def execute_with_feedback(task, context, prior_attempt, issue):
    system = "You complete exactly one task, correcting a specific flaw from a prior attempt."
    user = (
        f"Task: {task}\n\nContext:\n{context}\n\n"
        f"Prior attempt:\n{prior_attempt}\n\n"
        f"Problem with prior attempt: {issue}\n\nProduce a corrected version."
    )
    return call_llm(system, user)

Cap retries per step. Two or three attempts is usually enough; beyond that you're masking a prompt design problem, not fixing a transient issue. Log every failed attempt so you can look back and see which step in the chain fails most often, that step almost always needs its instructions tightened rather than more retries thrown at it.

Make steps idempotent where possible. If a step writes to a database or calls an external API, guard it so re-running the chain from a checkpoint doesn't duplicate side effects. Keep a run_id and a step_index, and have side-effecting steps check whether their output already exists before repeating the effect.

Checkpoint intermediate results. Persist each step's output (to a file, a database row, whatever's convenient) before moving to the next step. If step four fails, you resume from the saved output of step three instead of regenerating the whole chain, which saves both cost and time on long chains.

def run_chain_checkpointed(request, checkpoint_path):
    state = load_checkpoint(checkpoint_path) or {"tasks": None, "results": []}

    if state["tasks"] is None:
        state["tasks"] = json.loads(plan_step(request))
        save_checkpoint(checkpoint_path, state)

    completed = {r["task"] for r in state["results"]}
    context = "\n\n".join(f"[{r['task']}]\n{r['output']}" for r in state["results"])

    for task in state["tasks"]:
        if task in completed:
            continue
        output = execute_step(task, context)
        state["results"].append({"task": task, "output": output})
        context += f"\n\n[{task}]\n{output}"
        save_checkpoint(checkpoint_path, state)

    return state

A Worked Example: Research-to-Report Pipeline

Put the pieces together with a concrete case: turning a raw dump of product notes into a structured comparison report.

def research_to_report(product_notes, request):
    # Step 1: plan
    tasks = json.loads(plan_step(request))
    # e.g. ["extract feature list per product", "extract pricing model per product",
    #       "write comparison table description", "write recommendation section"]

    # Step 2: extraction sub-steps run first, structured
    features = extract_with_retry(product_notes)  # schema-specific prompt per field set

    # Step 3: writing sub-steps consume the structured extraction, not raw notes
    context = json.dumps(features)
    comparison = execute_step("write comparison section", context)
    recommendation = execute_step("write recommendation section", context + comparison)

    combined = comparison + "\n\n" + recommendation

    # Step 4: verify against the original ask
    verdict = json.loads(verify_step(request, combined))
    if not verdict["pass"]:
        for issue in verdict["issues"]:
            combined = execute_with_feedback("revise report", context, combined, issue)

    return combined

Notice the writing steps never see the raw, messy product notes, they only see the clean structured extraction. That single change (structured handoff instead of raw text handoff) is usually where the biggest quality jump in a decomposed pipeline comes from, because it removes an entire class of "the model misread the notes" errors.

Common Mistakes

Over-decomposing trivial tasks. Splitting a single classification call into a plan step, an execute step, and a verify step triples your latency and cost for a task the model already does correctly in one shot. Decompose based on evidence of failure, not by default.

Passing raw model output as the only handoff. If step two has to re-parse loosely formatted prose from step one to find the three numbers it needs, you've reintroduced the exact reliability problem decomposition was supposed to solve. Force structure at every handoff point.

Skipping the verify step because individual steps "looked fine." The whole point of decomposition is that local correctness doesn't guarantee global correctness. A pricing extraction can be accurate and a comparison write-up can be well-written, and together they can still contradict each other. Always check the combined output against the original request.

No bound on retries. An execute step stuck in a retry loop against a genuinely ambiguous instruction will burn tokens indefinitely if you don't cap attempts and fall back to a human review path.

Treating the plan step's output as gospel. The plan step is itself a model call and can produce a bad plan (missing a sub-task, ordering things wrong). Validate the plan's shape (right number of tasks, no duplicates, dependencies make sense) before executing it, the same way you'd validate any other step's output.

FAQ

Does task decomposition prompting always improve accuracy? No. It improves accuracy specifically when the original task bundles unrelated sub-goals or requires different reasoning modes in sequence. For a genuinely atomic task, decomposition adds latency without adding reliability, so measure before and after rather than assuming it always helps.

How many steps should a chain have? As few as the task's natural sub-goals require, typically two to five for most content and data-processing pipelines. If you're past six or seven steps, look for sub-tasks that can be merged or run in parallel rather than adding more sequential stages.

Should every step use the same model? No, match the model to the step's difficulty. A structured extraction or classification step can often run on a smaller, faster model, while the drafting or reasoning-heavy steps benefit from a stronger one. This also keeps overall latency and cost down since you're not paying premium-model rates for simple lookups.

How is this different from chain-of-thought prompting? Chain-of-thought asks a single model call to reason step by step within one response. Task decomposition prompting splits the work across multiple separate calls, each with its own prompt, context, and output validation. You can combine both: use chain-of-thought reasoning inside an individual execute step while still decomposing the overall task into a multi-call chain.

What's the simplest way to start decomposing an existing mega-prompt? Find the one sub-instruction inside it that fails most often (usually verification, arithmetic, or citation accuracy) and pull just that piece out into its own step with its own validation. You don't have to decompose the whole pipeline at once, isolating the weakest link first gives you most of the reliability gain for the least engineering effort.