ScholarLoop shows a cost-controlled pattern for research agents
ScholarLoop is an tool where several AI agents handle a reflow: reading papers, proposing ideas, running experiments, learning from results, and drafting a write-up. Its main design choice is to avoid spending heavily on every idea. It first tests many ideas with cheap early checks, then sends only the better ones into more expensive runs.
The loop can stop by budget limit, round limit, or lack of further improvement. Each agent’s prediction or judgment is compared with measured results, so later rounds can rely more on agents that were right before. Experiment scores use fixed measurement rules, and code-level guards try to stop fake gains or invented numbers.
Current examples cover digit and diabetes , with published runs showing small improvements over at about $0.45 and $0.77. The project has 108 tests, and its quick demo can run without an API key or GPU by using a fixed mock LLM.
Key points
- Multiple AI agents split the reflow into reading, proposing, testing, reflecting, and writing.
- Cheap early checks remove weak ideas before expensive runs happen.
- The system can stop automatically when it hits a budget, a round limit, or no further progress.
- Agent judgments are checked against real results and used to guide later rounds.
- A quick demo runs without an API key or GPU, and real run cost examples are published.