RAG Access Control and Vector Store Security

Filter before the prompt, not after

A RAG assistant searches your documents and gives the best passages to the model. If the search index covers everyone's documents and nothing checks who is asking, a user can retrieve (and the model can quote) a document they have no right to see: salaries, other customers' records, confidential plans. The fix is access control inside retrieval: attach permissions (tenant, groups, document ACLs) to each chunk as metadata and apply a filter for the current user in the search query itself, so forbidden chunks never reach the prompt. Do not retrieve first and hope the model withholds the text; once text is in the prompt, it can leak. Also protect the vector store (authentication, network isolation, encryption), remember that embeddings can leak information about their source text, and keep the index in step with deletions and permission changes.

Filter first versus no filter, run

I ran this with plain Python 3 (standard library only). All attacks here are harmless demonstrations on local data, using no real systems. A salary question retrieves documents 1, 2 and 3. With filtering inside retrieval an engineer receives only 1 and 3; without filtering the HR-only salary document 2 enters the prompt. An HR user legitimately gets all three.

DOCS = [
    {"id": 1, "text": "Public refund policy", "allowed": {"everyone"}},
    {"id": 2, "text": "Salary bands (HR only)", "allowed": {"hr"}},
    {"id": 3, "text": "Engineering roadmap", "allowed": {"eng", "hr"}},
]
def retrieve(query_hits, user_groups, filter_first):
    if filter_first:                                           # enforce access control INSIDE retrieval
        visible = [d for d in DOCS if d["allowed"] & (user_groups | {"everyone"})]
        return [d["id"] for d in visible if d["id"] in query_hits]
    return query_hits                                          # MISTAKE: everything retrieved goes into the prompt

hits = [1, 2, 3]                                               # what similarity search found for a salary question
print("engineer, filter first :", retrieve(hits, {"eng"}, True))
print("engineer, no filter    :", retrieve(hits, {"eng"}, False), "  <- salary document leaks into the prompt")
print("hr user, filter first  :", retrieve(hits, {"hr"}, True))

Output:

engineer, filter first : [1, 3]
engineer, no filter    : [1, 2, 3]   <- salary document leaks into the prompt
hr user, filter first  : [1, 2, 3]

Test with two users

Automate a check that user A can never retrieve user B's documents.

Quick check: Where must RAG permissions be enforced?

• Nowhere
• Only by asking the model to hide secrets
• Only in the UI
• In the retrieval query, so forbidden text never enters the prompt