Building the RAG Prompt

Rules, evidence, question

A RAG prompt has three parts. Instructions: answer only from the context, cite sources, say "I don't know" if the answer is missing, and follow the output format. Context: the chunks, each with a number or ID and, if useful, its source and date. Question. Put the instructions first (or in the system prompt) and the question last. Mark the context clearly as data, not instructions, since retrieved text can contain prompt-injection attempts. Ask for structured output (for example JSON with answer and citations) if code will consume it.

Prompt, cite, verify

A good prompt, citations you check in code and a groundedness test keep answers honest.

Assembling the prompt, run

I ran this plain-Python (standard library only) example. The function numbers each chunk so the model can cite [1] or [2].

def build_prompt(question, chunks):
    context = "\n".join(f"[{i}] {c}" for i, c in enumerate(chunks, start=1))
    return (
        "Answer only from the numbered context. Cite sources like [1]. "
        "If the answer is not in the context, say you do not know.\n\n"
        f"Context:\n{context}\n\nQuestion: {question}\nAnswer:"
    )

chunks = ["Employees get 24 days of paid leave per year.",
          "Unused leave up to 5 days can be carried over."]
print(build_prompt("How many days can I carry over?", chunks))

Output:

Answer only from the numbered context. Cite sources like [1]. If the answer is not in the context, say you do not know.

Context:
[1] Employees get 24 days of paid leave per year.
[2] Unused leave up to 5 days can be carried over.

Question: How many days can I carry over?
Answer:

Put the question last

Models tend to weight the end of the prompt strongly, so place the question after the context and repeat key rules briefly if the context is long.

Quick check: Why give each chunk a number or ID in the prompt?

• To shrink tokens
• To hide the text
• So the model can cite which chunk supports each claim
• To skip retrieval