Agent safety needs more than checking the words
AI agents that can use real tools can get past s that only inspect the request text. A public security can be turned into a sequence of tool calls, then rewritten by an LLM as a normal-looking request. The dangerous part may not appear in the words; it appears when the agent follows the tool-call path.
In tests with agents using MCP file-system tools, base models from 1B to 1 refused no more than 35% of these attacks. Safety training methods such as DPO and SafeDPO raised refusal to only 48%. Some methods worked better, with one reaching about three times the baseline refusal rate without another run.
The methodology, training and code, dataset, and papers are available.
Key points
- Text-only s can fail when AI agents use external tools.
- A request can look harmless while leading to a dangerous sequence of tool calls.
- With MCP file-system access, base models refused at most 35% of the tested attacks.
- DPO and SafeDPO improved refusal to 48%, but still missed more than half.
- One method reached about three times the baseline refusal rate.