Grounding and Hallucination Control

Answer from evidence

The best defence against hallucination is grounding: give the model the relevant source text (retrieval), instruct it to answer only from that text and to say when the answer is not there, and ask it to cite the passage. Then add a check that compares the answer with the sources. Even a crude check, such as how many of the answer's words appear in the source, can flag sentences that introduce new claims. Stronger checks use a second model or entailment classifier, with their own error rates.

A crude support score, run

I ran this. The first sentence is fully covered by the source (1.0); the second adds "water damage and theft" that the source never mentions, so only half its words are supported (0.5). This is a teaching toy, not a production check.

import re
src = "The warranty lasts 12 months and covers manufacturing defects only."

def support(sentence, source):
    w = set(re.findall(r"[a-z0-9]+", sentence.lower()))
    s = set(re.findall(r"[a-z0-9]+", source.lower()))
    return round(len(w & s) / len(w), 2)

for sent in ("The warranty lasts 12 months.",
             "The warranty also covers water damage and theft."):
    print(support(sent, src), sent)

Output:

1.0 The warranty lasts 12 months.
0.5 The warranty also covers water damage and theft.

Allow "I don't know"

Models tend to answer even when they should not. Say explicitly in the instructions that a refusal to guess is correct, and test with questions whose answers are not in the documents.

Quick check: What is a good instruction for reducing hallucination in a document Q&A bot?

• Guess when unsure
• Always sound confident
• Never mention sources
• Answer only from the provided text and say if it is not there