An AI agent can pass checks and still update the wrong customer
The risk with is not only that they might do something obviously forbidden. A harder problem appears when the agent is d, the API key has , and the policy check passes, but the action targets the wrong thing.
For example, an agent may send a document approval email, wait for a reply, see “approved,” and then update a database record. may allow the write, and the business policy may allow updates.
But if the reply came from a forwarded email thread and actually refers to another customer, the rules all pass while the wrong customer record is changed. An early of a is being built to check the agent’s , the business policy, and whether the source and context match before the agent runs a tool, API, or database action.
Key points
- An agent can pass and policy checks but still act on the wrong target.
- The risky target could be the wrong customer, tenant, ticket, row, or workflow.
- Forwarded email threads can make an approval look valid while pointing to another customer.
- A can check , policy, source, and context before execution.