AI agent benchmarks can cost thousands for a single run
Testing an AI agent on popular can create very large . such as and SWE-bench may cost thousands of dollars for just one run. Reliable results usually need several runs, because a single run may not show the average .
One estimate says running with Opus 4.6 could cost up to $40,000 for one run. Cheaper are needed if teams want useful results without spending close to $200,000.
Key points
- Popular AI agent can create very high .
- and SWE-bench are named as examples that may cost thousands of dollars per run.
- Multiple runs are often needed to get a more reliable average result.
- One estimate says with Opus 4.6 could cost up to $40,000 for a single run.
- Lower-cost tests can help teams reduce AI agent development costs.