REAP turns real coding-agent use into automatic benchmarks
REAP is presented as a way to automatically build for from real interactive . The main idea is to turn the work a already sees in live use into evaluation material, instead of relying only on hand-made test tasks. The available information does not confirm numbers, cost savings, setup steps, or whether the code is .
Key points
- REAP is about automatically creating .
- The source material comes from interactive .
- The goal appears to be reducing manual work in making evaluation tasks.
- It is more relevant to agent evaluation than to direct token savings.