RAG permission filters can become the real security line
An summarized a document that belonged to another customer. This was not an outside break-in; the returned the most similar text chunks it was asked to find. can judge similarity, but it does not know who is allowed to see each document.
The real problem was the design. Rules for workspace, owner, , role, and archived status were written by hand inside app code, with slightly different copies across three features. This filter ran on every retrieval and acted as the , but it had almost no testing.
When one condition was missed, nothing crashed; the AI produced a smooth, confident answer using data the person should not have seen. Later review was also hard because logs showed what the app requested, not the exact permission rules enforced for each query.
Key points
- finds similar content, but it does not understand access rights by itself.
- Copying logic across features makes small differences and missed rules more likely.
- The is effectively the for each retrieval step.
- A failed rule may produce a normal-looking AI answer instead of an error.
- Logs should record which permission rules were enforced for each query, not only what the app requested.