RAG for Onboarding Docs: Turning Your Handbook Into a Chatbot

Why Your Onboarding Docs Are a RAG Problem Waiting to Happen

Every company has one: a 40-page handbook living in Notion or Google Docs, last meaningfully updated eighteen months ago, that new hires are told to "read before your first day." Nobody reads it. Instead, they Slack a teammate and ask "how do I set up my VPN" or "what's our PTO policy" for the fifth time this quarter. The knowledge exists. It's just locked in a format humans don't query well — long-form prose meant to be read top to bottom, not searched for a single fact under time pressure.

This is exactly the shape of problem Retrieval-Augmented Generation was built to solve. RAG lets you keep your source documents as the single source of truth while giving employees a conversational interface on top of them. Instead of rewriting your handbook as a rigid FAQ or building a brittle keyword search box, you embed the content, retrieve the relevant chunks at query time, and let a language model synthesize an answer grounded in your actual policies — not the model's general training knowledge, which might confidently invent a PTO policy your company doesn't have.

This article walks through building a working onboarding chatbot from a handbook: chunking strategy, embeddings, retrieval, and the guardrails that keep it from hallucinating HR policy. If you're new to the underlying architecture, our Introduction to RAG course covers the fundamentals in more depth — this piece is the applied, "ship it this week" version aimed at one specific, very common use case.

What Makes Onboarding Docs a Distinct RAG Use Case

Not all document sets are equal, and onboarding handbooks have quirks that change your design choices compared to, say, a RAG system over legal contracts or product documentation.

• High factual stakes, low tolerance for hallucination. A wrong answer about health insurance enrollment deadlines or expense reimbursement isn't a minor inconvenience — it can cost an employee money or cause a compliance headache. This pushes you toward tighter grounding and explicit "I don't know" behavior over confident guessing.
• Frequent small updates. Handbooks change constantly: a new remote-work policy, an updated holiday calendar, a revised expense limit. Your ingestion pipeline needs to handle incremental re-indexing, not just a one-time bulk load.
• Mixed document types. Onboarding knowledge rarely lives in one file. It's spread across a handbook PDF, a benefits provider's separate document, Slack channel pinned messages, an IT setup wiki page, and maybe a video transcript from a welcome session. Your retrieval layer has to unify these.
• Personalization needs. Not every answer applies to every employee. A contractor asking about 401(k) matching should get a different answer than a full-time employee. Your RAG system either needs metadata filtering or the humility to say "this depends on your employment type — check with HR."
• Low query volume, high embarrassment cost. Unlike a customer-facing support bot handling thousands of queries a day, an internal onboarding bot might get a few hundred queries a week. But if it gives a new hire wrong information in their first week, that's a trust hit that's hard to recover from. This argues for investing more per-query effort (better retrieval, maybe a reranking step) rather than optimizing purely for cost and latency.

Understanding these constraints upfront shapes almost every downstream decision, from chunk size to which retrieval strategy you pick.

Step 1: Getting the Handbook Ready for Ingestion

Before you touch an embedding model, you need clean, structured text. This is the least glamorous part of RAG and the part most tutorials skip — but it's where most real-world quality problems originate.

Start by exporting your handbook out of whatever tool it lives in (Notion, Confluence, Google Docs) into plain Markdown or HTML. Preserve headings — they're your best free signal for chunk boundaries and metadata. A handbook section titled "## Parental Leave" is gold for retrieval if you keep that heading attached to its content as metadata, because you can use it for both filtering and citation.

Here's a simple ingestion script that walks a directory of Markdown files, splits them by heading, and produces chunk objects with metadata attached:

import re
from pathlib import Path
from dataclasses import dataclass, field

@dataclass
class Chunk:
    text: str
    source_file: str
    section_title: str
    chunk_id: str
    metadata: dict = field(default_factory=dict)

def split_by_heading(markdown_text: str, source_file: str):
    # Split on level-2 headings, keep the heading with its body
    pattern = r"(?=^## )"
    sections = re.split(pattern, markdown_text, flags=re.MULTILINE)
    chunks = []
    for i, section in enumerate(sections):
        if not section.strip():
            continue
        lines = section.strip().split("\n")
        title = lines[0].replace("## ", "").strip() if lines[0].startswith("## ") else "Introduction"
        body = "\n".join(lines[1:]).strip()
        if len(body) < 20:
            continue
        chunks.append(Chunk(
            text=f"{title}\n{body}",
            source_file=source_file,
            section_title=title,
            chunk_id=f"{source_file}::{i}",
            metadata={"doc_type": "handbook", "employee_types": ["full_time", "contractor"]}
        ))
    return chunks

def ingest_handbook_directory(directory: str):
    all_chunks = []
    for path in Path(directory).glob("*.md"):
        text = path.read_text(encoding="utf-8")
        all_chunks.extend(split_by_heading(text, path.name))
    return all_chunks

if __name__ == "__main__":
    chunks = ingest_handbook_directory("./handbook_docs")
    print(f"Produced {len(chunks)} chunks from handbook")
    for c in chunks[:3]:
        print(f"- [{c.section_title}] {c.text[:80]}...")

A few things worth calling out here. First, splitting on ## headings rather than a fixed character count keeps semantically related content together — a policy explanation doesn't get sliced in half mid-sentence. Second, the metadata field is doing real work: employee_types lets you later filter out a contractor-only clause when a full-time employee asks a question, or vice versa. Third, we discard tiny sections (under 20 characters) since a lone heading with no body is noise, not a retrievable unit.

If your source is a PDF instead of Markdown, run it through a PDF-to-text extractor that preserves layout reasonably well, then apply the same heading-based split. Tables inside PDFs (common for benefits comparison charts) are the hardest part — you'll often get better results extracting them separately and converting to a plain-text description rather than trying to embed a mangled table dump.

Step 2: Chunking Strategy for Policy-Heavy Text

Generic RAG tutorials often recommend fixed-size chunking — say, 512 tokens with 50 tokens of overlap. That's a reasonable default for narrative text, but onboarding docs benefit from a more structure-aware approach for two reasons: policies are often short and self-contained (a PTO accrual rule might be three sentences), and losing the heading context loses the "what is this actually about" signal that helps both retrieval and the generated answer.

Practical guidelines that work well for handbook content:

• Chunk by section, not by fixed token count, when your source has clear headings. Cap section chunks that run long (over ~600 tokens) by splitting on paragraph boundaries within the section, but keep the section title prepended to every sub-chunk.
• Keep chunks between 150 and 500 tokens. Shorter than that and you lose context ("this applies only if you were hired before 2023" might land in a different chunk than the rule it qualifies). Longer than that and retrieval precision drops — you're returning a wall of text and asking the LLM to find the one relevant sentence.
• Always prepend the section title and, ideally, a breadcrumb (e.g., "Handbook > Benefits > Parental Leave") to the chunk text before embedding. This means the embedding captures topical context even if the body text is terse, and it gives you a natural citation to show the user.
• Overlap sparingly. A 10-15% overlap between adjacent chunks within the same long section helps avoid losing content that straddles a split point, but don't overlap across section boundaries — that just duplicates noise.
• Store an "effective date" or "last updated" field per chunk if you can get it. Handbooks get revised, and if your vector store still has stale chunks from an old PTO policy alongside new ones, retrieval might surface both. A recency filter or metadata boost avoids the bot confidently citing an outdated rule.

Step 3: Building the Embedding and Retrieval Pipeline

With clean chunks in hand, the next step is embedding them into a vector store. For an onboarding bot with a few hundred to a few thousand chunks, you don't need a heavyweight distributed vector database — something like a local FAISS index or a managed pgvector table is plenty.

import numpy as np
from openai import OpenAI

client = OpenAI()

def embed_texts(texts: list[str], model="text-embedding-3-small") -> list[list[float]]:
    response = client.embeddings.create(input=texts, model=model)
    return [item.embedding for item in response.data]

class SimpleVectorStore:
    def __init__(self):
        self.vectors = []
        self.chunks = []

    def add(self, chunks, embeddings):
        self.chunks.extend(chunks)
        self.vectors.extend(embeddings)

    def search(self, query_embedding, top_k=5, employee_type=None):
        scores = []
        for i, vec in enumerate(self.vectors):
            chunk = self.chunks[i]
            if employee_type and employee_type not in chunk.metadata.get("employee_types", []):
                continue
            sim = np.dot(query_embedding, vec) / (
                np.linalg.norm(query_embedding) * np.linalg.norm(vec)
            )
            scores.append((sim, chunk))
        scores.sort(key=lambda x: x[0], reverse=True)
        return scores[:top_k]

def build_index(chunks):
    store = SimpleVectorStore()
    texts = [c.text for c in chunks]
    embeddings = embed_texts(texts)
    store.add(chunks, embeddings)
    return store

def retrieve(store, query: str, employee_type=None, top_k=5):
    query_embedding = embed_texts([query])[0]
    results = store.search(query_embedding, top_k=top_k, employee_type=employee_type)
    return results

This is intentionally minimal — no external vector database dependency — so you can see exactly what's happening: embed the chunks once at ingestion time, embed the query at request time, and rank by cosine similarity, with an optional metadata filter for employee type layered on top. For production you'd swap SimpleVectorStore for pgvector or a managed store, but the retrieval logic doesn't fundamentally change.

One detail that matters more than people expect: use the same embedding model for both ingestion and query time, and re-embed everything if you ever switch models. Mixing embeddings from text-embedding-3-small and an older model will silently degrade retrieval quality with no obvious error — the cosine similarities will just be meaningless.

Step 4: Generating Grounded Answers

Retrieval gets you the right chunks; generation turns them into a helpful, accurate answer. The prompt template is where you enforce the "don't hallucinate HR policy" behavior that matters so much for this use case.

SYSTEM_PROMPT = """You are an onboarding assistant for new employees. Answer questions
using ONLY the context provided below. If the context does not contain enough
information to answer confidently, say so and suggest the employee contact HR
directly — do not guess or use outside knowledge about typical company policies.

Always cite which section your answer came from, like this: (Source: Benefits > Parental Leave)
"""

def build_prompt(query: str, retrieved_chunks) -> str:
    context_blocks = []
    for score, chunk in retrieved_chunks:
        context_blocks.append(
            f"[Source: {chunk.section_title}]\n{chunk.text}"
        )
    context = "\n\n---\n\n".join(context_blocks)
    return f"""Context:
{context}

Question: {query}

Answer using only the context above. Cite the source section."""

def answer_question(client, store, query: str, employee_type=None):
    results = retrieve(store, query, employee_type=employee_type, top_k=5)
    if not results or results[0][0] < 0.3:
        return "I couldn't find anything relevant to that in the handbook. Please check with HR directly."

    prompt = build_prompt(query, results)
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": prompt},
        ],
        temperature=0.1,
    )
    return response.choices[0].message.content

Two design choices here directly address the hallucination risk that's the biggest failure mode for this use case. First, the similarity threshold check (results[0][0] < 0.3) refuses to even call the LLM if nothing relevant was retrieved — this catches out-of-scope questions like "what's the WiFi password at the Austin office" when your handbook only covers HR policy. Second, low temperature (0.1) keeps the model close to the source text instead of getting creative with phrasing that might drift from the actual policy language.

The citation instruction matters more than it looks. New hires trust an answer more when they can see exactly where it came from, and it gives them a natural next step — "let me go read that whole section" — instead of treating the bot's paraphrase as the final word on a policy that might have important caveats the chunk didn't fully capture.

Handling the Update Problem

The single biggest reason internal RAG bots go stale and get abandoned is that nobody wires up re-ingestion when the source docs change. If HR updates the parental leave policy in Notion, your vector store needs to reflect that within a day, not whenever someone remembers to re-run the ingestion script.

A few practical patterns:

1. Webhook-triggered re-ingestion. Most doc platforms (Notion, Confluence, Google Docs via Apps Script) can fire a webhook on page update. Have that webhook hit an endpoint that re-chunks and re-embeds just the changed document, then swaps out its old chunks in the vector store by source_file key.
2. Scheduled full re-sync as a fallback. Even with webhooks, run a nightly job that re-pulls everything and diffs against what's indexed, in case a webhook was missed.
3. Version the chunks. Store an ingested_at timestamp and a content hash per chunk. If the hash hasn't changed, skip re-embedding — this saves API cost as your handbook grows.
4. Delete, don't just add. When a section is removed or rewritten, make sure old chunks for that section are deleted from the index, not left to compete with the new ones during retrieval. Stale duplicate chunks are a quiet but common cause of a bot that "sometimes gives the old answer."

Evaluating Whether It's Actually Working

Before rolling this out to real new hires, build a small evaluation set — 20 to 30 realistic questions with known-correct answers pulled straight from the handbook. Questions like "how many sick days do I get in my first year" or "what's the process for expensing a conference" are exactly the kind of thing new hires ask in week one.

Run each question through the pipeline and check three things: did retrieval surface the correct source section, did the generated answer match the handbook's actual content, and did the bot correctly refuse when asked something outside its knowledge (try a trick question like "what's my manager's cell phone number" to confirm it doesn't fabricate). This evaluation set doubles as a regression test — rerun it whenever you change the embedding model, chunking strategy, or prompt, since small changes to any of these can shift retrieval quality in ways that aren't obvious from casual testing.

It's also worth tracking real usage once it's live: log every query and which chunks were retrieved, then periodically review the queries that got a low-confidence or "I don't know" response. These are your signal for gaps in the handbook itself, not just gaps in the bot — often the fix isn't better retrieval, it's writing the missing policy section in the first place.

Rolling It Out Without Overpromising

Set expectations clearly when you launch this internally. Frame it as "ask the handbook" rather than "ask HR" — the bot should feel like a faster way to search documentation, not a replacement for a human who can handle edge cases, exceptions, and anything emotionally sensitive (medical leave, harassment complaints, compensation disputes should always route to a human, and it's worth hard-coding a refusal for categories like these regardless of what the retrieval turns up).

A simple, effective pattern is a hybrid handoff: the bot answers straightforward policy questions confidently, and for anything it flags as low-confidence, ambiguous, or sensitive, it responds with a short escalation message and a link to open a ticket with HR. This keeps trust high because the bot is never the last word on something it shouldn't be deciding, while still handling the 80% of repetitive questions that used to eat up a people-ops team's time.

Start with a narrow rollout — one team, one cohort of new hires — before opening it company-wide. Watch the query logs closely in the first two weeks. You'll almost certainly find phrasing patterns your chunking missed (employees asking about "vacation" when your handbook says "PTO," for instance) that are cheap to fix once you see them but easy to miss in a synthetic test set.

Where to Go From Here

Turning a static handbook into a chatbot is a genuinely good first RAG project because the stakes are real but contained, the document set is small enough to iterate on quickly, and the failure modes — hallucinated policy, stale answers, missing citations — are exactly the ones you'll run into in any production RAG system, just with lower blast radius than a customer-facing product.

Once this is working well, the same architecture extends naturally: onboarding IT setup guides, engineering runbooks, sales playbooks — any internal knowledge base that's currently trapped in documents nobody reads cover to cover is a candidate for the same pipeline. If you want to go deeper on the retrieval theory, evaluation techniques, and architectural variants (hybrid search, reranking, multi-hop retrieval) that this article only touched on, our Introduction to RAG course builds up from these same fundamentals into the more advanced patterns you'll need as your document set and query volume grow.