What RAG Does Not Fix

Better, not perfect

RAG reduces hallucination but does not remove it. Common failures: the right passage is never retrieved (the most common cause of bad answers); the passage is retrieved but ignored or misread; retrieved text is outdated or wrong (garbage in, garbage out); the answer needs combining many documents (multi-hop questions) or calculating across a table; the question is ambiguous; and conflicting sources disagree. Good RAG engineering is mostly about measuring which of these is happening and fixing that stage.

Always allow "I don't know"

Instruct the model to say it does not know when the context lacks the answer, and measure how often it does so correctly.

Quick check: What is the most common cause of bad RAG answers?

• The font was too small
• The relevant passage was not retrieved
• The GPU was cold
• The answer was too short