Building a Research Agent That Cites Its Sources

Ask any large language model a research question and it will answer instantly, fluently, and sometimes wrong. Not obviously wrong — subtly wrong, in the way that sounds correct until someone checks. The gap between "sounds authoritative" and "is actually true" is exactly where research agents live or die. If you have ever pasted an LLM's summary into a report and then spent twenty minutes trying to verify a statistic it invented, you already understand why citations are not a nice-to-have feature. They are the entire point. A research agent without sources is just a confident guesser with better grammar. In this article we will build one that behaves differently: it searches, it reads, it tracks exactly where every claim came from, and it refuses to state something it cannot point back to. We will go from a naive single-shot prompt to a multi-step agent with retrieval, citation binding, and verification passes, with working code you can adapt today.

Why "just ask the model" fails for research

The core failure mode of an uncited research answer is that fabrication and fact are indistinguishable at the surface level. Both come out as fluent, confident prose. A model that read a real source and a model that hallucinated a similar-sounding fact will phrase their sentences almost identically. There is no linguistic tell. The only way to separate them is to force the model to show its work in a structure that can be independently checked — a URL, a document ID, a page number, something a human or a second system can go verify.

There is a second, quieter failure mode: even when a model paraphrases a real source correctly, drift happens over multi-turn conversations. Ask a follow-up question three turns later and the model may blend the original source with something adjacent from its training data, producing a statement that is half-grounded and half-invented, with no way to tell which half is which. Citations solve this by anchoring every atomic claim to a specific retrieved passage at generation time, not to the model's general knowledge.

The practical implication for engineering is that "add citations" cannot be a final polish step where you ask the model to bolt links onto an already-written answer. That produces citation theater — links that exist but do not actually support the sentence next to them. Real grounding has to be structural: the agent fetches sources first, the answer is built by quoting and referencing those sources directly, and a verification step checks that the final claims and the final citations actually match.

Architecture: the four stages of a citation-grounded agent

A research agent that cites sources reliably tends to separate into four distinct stages, each with a narrow job:

• Query planning: decompose the user's question into one or more concrete search queries
• Retrieval: run those queries against a search tool and fetch full content for promising results
• Grounded synthesis: generate an answer where each claim is written alongside the source it came from
• Verification: check that every claim in the final answer is actually supported by its attached citation

This is deliberately more steps than a single prompt-and-respond loop. The temptation is to let the model do search and writing in one pass because it is faster to build. Resist it. The moment synthesis and retrieval are the same step, you lose the ability to check the model's claim against the source text programmatically, and you are back to trusting fluency as a proxy for truth.

Here is the shape of the pipeline in code, using a simple tool-calling loop that should map onto any LLM SDK that supports function calling:

from dataclasses import dataclass, field

@dataclass
class Source:
    id: str
    url: str
    title: str
    content: str

@dataclass
class Claim:
    text: str
    source_ids: list[str] = field(default_factory=list)

class ResearchAgent:
    def __init__(self, llm, search_tool, fetch_tool):
        self.llm = llm
        self.search_tool = search_tool
        self.fetch_tool = fetch_tool
        self.sources: dict[str, Source] = {}

    def run(self, question: str) -> dict:
        queries = self.plan_queries(question)
        for query in queries:
            self.retrieve(query)
        claims = self.synthesize(question)
        verified = self.verify(claims)
        return self.format_report(verified)

Each method in that skeleton is a separate concern with a separate failure mode, which is exactly why we keep them apart. plan_queries can fail by asking vague questions. retrieve can fail by fetching low-quality pages. synthesize can fail by misattributing a claim to the wrong source. verify can fail by being too lenient. Debugging a monolithic prompt means guessing which of these four things went wrong from the output alone. Debugging a staged pipeline means you can log the output of each stage and see precisely where grounding broke.

Stage one: turning a question into searchable queries

Most real questions are not good search queries. "How has remote work affected mid-sized SaaS company retention in the last two years" is a research question, not something you type into a search box and expect a single decisive page back. The planning stage's job is to decompose the question into several narrower, independently searchable queries, and to do so explicitly rather than leaving it implicit in the model's head.

PLAN_PROMPT = """You are a research planner. Break the user's question into
2-4 concrete, narrow search queries that together would let someone answer
the full question using real, checkable sources. Avoid vague queries.

Question: {question}

Return a JSON list of strings, nothing else."""

def plan_queries(self, question: str) -> list[str]:
    response = self.llm.complete(PLAN_PROMPT.format(question=question))
    queries = parse_json_list(response)
    if not queries:
        # Fallback: treat the raw question as a single query rather
        # than silently returning nothing to search
        queries = [question]
    return queries[:4]

The cap on query count is deliberate. An agent that generates twelve queries for every question will drown its own context window in low-relevance pages by the time it reaches synthesis, and more sources does not mean more grounded — it often means more noise for the synthesis step to misattribute. Four focused queries beat twelve scattershot ones.

Stage two: retrieval that keeps source identity intact

This is the stage most implementations get subtly wrong. It is not enough to run a search and hand the model a blob of snippets. Every single piece of text that could end up in the final answer needs a stable identifier attached to it before it ever reaches the synthesis prompt. If you let the model see search results as an undifferentiated wall of text, it will happily blend two sources into one claim, and there will be no way afterward to tell which source it actually meant.

def retrieve(self, query: str, max_results: int = 4) -> None:
    results = self.search_tool.search(query, limit=max_results)
    for result in results:
        if result.url in self._seen_urls():
            continue
        full_text = self.fetch_tool.fetch(result.url)
        source_id = f"s{len(self.sources) + 1}"
        self.sources[source_id] = Source(
            id=source_id,
            url=result.url,
            title=result.title,
            content=full_text[:8000],  # cap per-source content
        )

def _seen_urls(self) -> set[str]:
    return {s.url for s in self.sources.values()}

Two details matter here beyond the obvious. First, deduplicating by URL prevents the same source from acquiring two different IDs across different queries, which would otherwise let the model cite "two sources" that are actually one page, inflating apparent corroboration. Second, truncating content per source is a practical necessity — full pages can be enormous, and an agent that stuffs ten full articles into one prompt will blow its context budget before it even starts writing. Truncate deliberately (keep the opening and any sections matching the query terms) rather than just chopping at a fixed character count with no regard for where the useful material sits.

Stage three: synthesis that binds every claim to a source ID

Synthesis is where citation quality is actually decided, and the trick is to make the source IDs part of the model's working vocabulary rather than an afterthought. Instead of asking the model to "write an answer and add citations," ask it to produce claims as structured objects, each one paired with the ID of the source it came from. This forces the model to make the attribution decision explicitly, at the moment it writes the claim, rather than retrofitting citations onto prose it already committed to.

SYNTHESIS_PROMPT = """Using ONLY the sources below, answer the question.
Every factual claim must be tagged with the source ID it came from, using
the format [[source_id]] immediately after the claim. If the sources do
not support a claim, do not make it — say what is missing instead.

Sources:
{source_block}

Question: {question}

Write your answer as a JSON list of objects with keys "text" and
"source_ids" (a list, since a claim can be backed by more than one source).
"""

def synthesize(self, question: str) -> list[Claim]:
    source_block = "\n\n".join(
        f"[{s.id}] {s.title} ({s.url})\n{s.content}"
        for s in self.sources.values()
    )
    prompt = SYNTHESIS_PROMPT.format(source_block=source_block, question=question)
    raw = self.llm.complete(prompt)
    parsed = parse_json_list(raw)
    return [Claim(text=c["text"], source_ids=c.get("source_ids", [])) for c in parsed]

Notice the instruction "if the sources do not support a claim, do not make it." This single line does more work than almost anything else in the prompt. It gives the model explicit permission to say "the retrieved sources don't cover this" instead of filling the gap with plausible-sounding invention. Models default to being helpful, and unconstrained helpfulness in a research context means guessing rather than admitting a gap. Naming the alternative behavior in the prompt makes it a legitimate output rather than a failure to be helpful.

Stage four: verification — the step almost everyone skips

Here is the uncomfortable truth: a model that is instructed to cite its sources will still occasionally attach a citation to a claim the source does not actually support. It is not lying maliciously — it is pattern-matching "this sentence is near this source, so tag it," which is a weaker guarantee than "this source's text actually contains or implies this claim." Verification is a second, independent pass whose only job is to check that guarantee before the answer ever reaches the user.

VERIFY_PROMPT = """Does the following source text support this claim?
Answer with just YES or NO, then a one-line reason.

Claim: {claim}

Source text:
{source_text}
"""

def verify(self, claims: list[Claim]) -> list[Claim]:
    verified = []
    for claim in claims:
        if not claim.source_ids:
            # Unsourced claims are dropped, not silently kept
            continue
        supported = True
        for sid in claim.source_ids:
            source = self.sources.get(sid)
            if source is None:
                supported = False
                break
            result = self.llm.complete(
                VERIFY_PROMPT.format(claim=claim.text, source_text=source.content[:3000])
            )
            if not result.strip().upper().startswith("YES"):
                supported = False
                break
        if supported:
            verified.append(claim)
    return verified

This costs extra LLM calls, and it is worth every one of them. Verification is intentionally a separate call from synthesis, using a narrower prompt with a single yes-or-no question, because a focused check is far more reliable than asking the same model, in the same breath it wrote the claim, to also grade its own work. Self-grading in one pass tends to rubber-stamp; a fresh call with only the claim and the source text in front of it is a genuinely independent check.

A claim that fails verification should be dropped from the final report, not softened or kept with a caveat. Softening language ("it appears that...") without removing an unsupported claim just relocates the problem — the sentence is still in the report, still reads as informative, and still is not actually backed by anything. Drop it, and if the resulting answer feels thin, that thinness is honest information: it tells you the retrieval stage did not find enough to answer that part of the question, which is a signal to run another query round, not to paper over with unverified prose.

Formatting the final report so citations stay attached

Once claims are verified, the last engineering problem is presentation: making sure the citation survives the trip from data structure to readable text. A common mistake is generating the prose first and the source list separately, which is exactly how citation-text mismatches sneak back in at the last step. Build the report directly from the verified claim objects instead, so the mapping is mechanical rather than another opportunity for the model to improvise.

def format_report(self, claims: list[Claim]) -> dict:
    lines = []
    footnotes = {}
    for claim in claims:
        marks = "".join(f"[{sid}]" for sid in claim.source_ids)
        lines.append(f"{claim.text} {marks}")
        for sid in claim.source_ids:
            footnotes[sid] = self.sources[sid].url

    body = "\n".join(lines)
    references = "\n".join(f"[{sid}] {url}" for sid, url in sorted(footnotes.items()))
    return {"body": body, "references": references, "claim_count": len(claims)}

The claim_count field is small but useful in production: if a run returns zero verified claims, that is a strong signal to surface a "couldn't find reliable sources for this" message to the user rather than an empty or generic answer that looks like a normal response but silently contains nothing.

Handling the hard cases: conflicting sources and thin coverage

Two situations will break a naive version of this pipeline, and both are worth handling explicitly rather than discovering them in production.

• Conflicting sources: two retrieved pages disagree on a number or a date. The synthesis prompt should be told explicitly to surface disagreement rather than pick one silently — instruct it to write both claims with their respective source IDs and note the conflict in plain language, so the user sees "Source A says X, Source B says Y" instead of a single confident-but-arbitrary answer.
• Thin coverage: retrieval returns sources that are tangential to the question. The verification stage will correctly reject most claims drawn from them, which can leave a report that is mostly empty. Treat a low claim_count as a trigger to run a second retrieval round with reformulated queries before giving up, rather than returning a thin report as if it were complete.

Both of these are failure modes that only become visible because the pipeline is structured — in a single-shot prompt, conflicting sources get silently averaged into one plausible-sounding number, and thin coverage gets silently padded with the model's background knowledge. Structure does not just add citations; it exposes problems that an unstructured approach would have hidden.

A note on search tool choice and rate limits

The pipeline above is agnostic to which search API sits behind search_tool, and that is intentional — you can wire it to a web search API, an internal document index, or a vector database of your own corpus with no change to the synthesis or verification logic. What does change with the backend is how aggressively you should cap max_results and content length. A general web search will return more irrelevant pages than a curated internal index, so widen the query count and rely more heavily on verification to filter noise. An internal, pre-vetted corpus can usually run with fewer, larger fetches per query since relevance is already higher going in.

Whichever backend you choose, log every fetched URL and every verification decision to a persistent store, not just to stdout. When a user later asks "where did this claim come from," you want to answer from a log, not by re-running the whole pipeline and hoping it retrieves the same sources twice — search results are not guaranteed stable across time, and re-running is not the same as recalling.

Testing: treat hallucination as a regression, not a vibe

Because the whole point of this system is trustworthiness, it needs the same testing discipline as any other correctness-critical code, not "seems fine when I tried it a few times." Build a small fixture set of questions with known-good and known-bad source material, and assert on the structural properties, not just eyeballing the prose.

def test_unsourced_claims_are_dropped():
    agent = ResearchAgent(llm=StubLLM(), search_tool=StubSearch(), fetch_tool=StubFetch())
    agent.sources = {"s1": Source(id="s1", url="http://x", title="X", content="Revenue grew 12%.")}
    claims = [
        Claim(text="Revenue grew 12%.", source_ids=["s1"]),
        Claim(text="The CEO plans to retire next year.", source_ids=[]),
    ]
    verified = agent.verify(claims)
    assert len(verified) == 1
    assert verified[0].text == "Revenue grew 12%."

Run this kind of test against every prompt change. Citation grounding is exactly the sort of property that regresses quietly — a prompt tweak that improves fluency can simultaneously make the model looser about attaching real citations, and you will not notice from reading a handful of outputs. A test suite that specifically checks "unsourced claims get dropped" and "verification rejects unsupported claims" catches that regression before it reaches a user asking a question that actually matters to them.

Putting it together

The full loop — plan, retrieve, synthesize with bound citations, verify independently, format from verified data — is more code than a single prompt, and that is the tradeoff you are making on purpose. You are trading a faster build for a system where every sentence in the output can be traced back to a real, fetchable source, and where the failure mode when sources are thin is an honest "not enough evidence" rather than a fluent guess dressed up as fact. That tradeoff is exactly what separates a demo from something you can put in front of a user who is going to act on what it tells them.

If you want to go deeper into building agents like this — tool-calling loops, verification patterns, and the engineering discipline behind agents that are actually reliable rather than merely articulate — that is precisely the ground we cover, step by step, in 30 Days of Hermes Agent, our hands-on course on building production-grade AI agents from first principles.