Open-source proxy blocks what AI agents actually do, not just what they say
This project tackles a specific fear about AI agents: the danger isn't the model saying something dumb, it's the model actually executing something destructive, like an accidental file deletion with no undo. The tool works as a proxy sitting between an app and any LLM — changing one line (the base_url) routes every tool call through a policy check before it runs. Rules are written in YAML, for example blocking rm -rf commands or forcing dry_run (a preview mode that shows what would happen without actually doing it) on actions.
When a call gets blocked, the model is told why, and it usually reconsiders its approach instead of just failing. A CI evaluation reports a 90.9% attack catch rate (40 of 44 test attacks blocked) and a 5.0% (1 of 20 legitimate calls wrongly blocked). The 4 missed attacks are deliberately left in the test set: base64-encoded secrets and non-English attempts (attacks that trick a model into unintended actions).
The code is and a demo runs without needing an API key.
Key points
- Sits as a proxy between app and LLM, checking every tool call against policy before execution
- Rules written in YAML — e.g., block rm -rf, force dry_run on deploys
- 90.9% attack catch rate (40/44), 5.0% (1/20)
- Missed attacks: base64-encoded secrets and non-English
- , demo runs without an API key, code on GitHub