Real costs behind a self-improving AI agent
An agent uses three design choices beyond the usual agent loop. First, a learned change is kept only when a verified result and an A/B test show that the pass rate improved. The decision is based on the real , not on the model claiming it improved. This needs a grader and means the agent accepts fewer changes more slowly.
Second, is treated as something that cannot be fixed by better wording alone, so safety is built into the system. stops memory and skills created during risky runs from being promoted automatically, a quarantined reader turns untrusted content into checked fields before a more powerful agent sees it, and the becomes smaller when risk is detected. In red-team testing, ASR fell from 100% to about 14%, but the tradeoff is that some valid content may be restricted too much. Third, benchmark results include and failure cases, without repeating runs just to get better-looking numbers.
The reasoning core uses a fusion panel that creates, judges, and combines answers, with a cost-aware router choosing the path behind it. The project is released under Apache-2.0.
Key points
- A change is kept only after a verified result and an A/B test show a better pass rate.
- The system judges the real , not the model's own report about success.
- is handled with , a quarantined reader, and a narrower under risk.
- Red-team ASR dropped from 100% to about 14%, with the downside of over-restricting some valid content.
- The agent uses a fusion panel plus a cost-aware router to choose how much reasoning to spend.