On This Page
A citation points to a contract, but it belongs to the wrong customer. In this illustrative case, a support assistant searches two customers’ folders for a renewal date. It finds a similar contract from the other customer, places it in context, and answers fluently.
This is not a reported incident. The citation makes the answer look careful while revealing the wrong customer’s data. The defect happened before the model wrote a sentence.
An ingested talk on where AI knowledge should live makes a useful architectural point: knowledge that varies by user needs an access-controlled retrieval layer. Microsoft’s document-level access guidance describes permission checks tied to indexed documents and query-time identity. The general rule is simpler than any one search product: only authorized material should reach the answer path.
Put the boundary before the answer
A prompt can tell an assistant to be concise, cite sources or ask for clarification. It cannot safely say “remember not to use the wrong customer’s documents” while handing the model those documents. If retrieval has already crossed the boundary, the prompt is being asked to undo an access failure.
The caller’s identity needs to be established by the application. The search query must carry the resulting scope: tenant, role, account or document permissions as the domain requires. Candidate results should be filtered or authorized before their text is supplied to the model. If an answer could trigger a write, the write path must check authorization again against the target record.
- Caller identity
authenticate in the application
- Scoped retrieval tenant · role · document
filter before model context
- Grounded answer
cite permitted evidence
- Write authorization
recheck the target record
This is not only a vector-search problem. Keyword search, cached summaries, conversation memory and tool results can all carry private content. A summary derived from one tenant’s document is still private after the source text has been shortened.
Our guide to connecting business tools asks who owns each record and which system is authoritative. For an assistant, add another question: who is allowed to know this record exists?
A small example exposes the ordering
Suppose an illustrative index has four documents: two public help articles, one contract for Tenant A and one for Tenant B. A user from Tenant A asks about billing terms. The closest semantic match happens to be Tenant B’s contract.
An unscoped retrieval step returns it. A later instruction to “answer only for Tenant A” cannot guarantee safety; the content has already entered the model context. A scoped retrieval step removes Tenant B’s document before ranking or returns only authorized candidates. The assistant may then answer from Tenant A’s contract or say it lacks the needed source.
The exact implementation depends on the platform. Azure AI Search’s security-filter pattern attaches group identifiers to documents and applies a filter on each query. Its documentation also warns that hiding a permission field from the response is not itself access control. The enforceable part is the filter and the caller identity behind it.
The distinction matters when permissions change. If a worker loses access, an old cache or stale index must not continue serving the document. Teams need a defined update path and tests for revocation, not only tests for initial grants. If the retrieval service cannot honor the source’s access model, copying the source into that index may be the wrong architecture.
Permission and relevance are different tests
An authorized document can be irrelevant or outdated. An accurate answer can be unauthorized. These are separate failure modes, so evaluate them separately.
First test the boundary: can a caller from one tenant retrieve, cite or infer private content from another? Include direct questions, paraphrases and summaries. Then test answer quality inside the permitted set: does the assistant choose the current policy and admit when evidence is absent? Microsoft’s RAG guidance explicitly recommends access control at retrieval time and treating retrieved content as untrusted.
There is a cost. Permission-aware indexing requires metadata, identity mapping and revocation handling. Some search results will become empty after filtering. That is a correct outcome when the caller lacks evidence. Filling the gap with a confident guess would turn an access safeguard into an accuracy problem.
This boundary is also a practical test of the broader automation exception rule: what should happen when the authorized search set is empty? A good design can return “I cannot access a source for that answer” and offer the right escalation path. It should not search a wider tenant, reuse an old answer or infer private details from a document title. Treat that empty result as a normal, testable state of the workflow. Permission is part of correctness, not a special case to patch after launch.
A useful assistant is not one that knows everything in the company. It is one that can find the right evidence within the caller’s authority and say when that boundary leaves the question unanswered.
Frequently Asked Questions
Can a prompt tell an assistant not to reveal private records?
A prompt can describe behavior, but it cannot replace authorization. Restrict retrieval to documents the caller may access and enforce permission again before consequential actions.
Where should tenant permissions be applied in a retrieval system?
Bind the authenticated caller to the query and filter or authorize candidate documents before their contents enter the model context. Keep permission metadata accurate as access changes.
Is a citation enough to make an answer trustworthy?
No. A citation shows where an answer came from. It does not prove the caller was allowed to see the source or that the underlying data is current and correct.