UC Berkeley ALE benchmark shows AI still struggles with real work
UC Berkeley’s new tests whether AI models can finish useful real-world tasks across 13 industries and 55 fields. The results are weak across the board. This is not like many coding benchmarks where models can score above 90%; every model struggles more when the work is closer to real business tasks.
Cost depends heavily on the harness, not just the model. Common standard harnesses often do poorly on cost efficiency. Chinese models are improving on other benchmarks, but in this test their success rates are about half of .
Claude performs much worse here than it does on narrower tests, and in one case it costs almost 10 times as much as competing models with similar or slightly better .
Key points
- The tests real tasks across 13 industries and 55 fields.
- All tested models struggle more than they do on many coding benchmarks.
- The harness can strongly change total cost and efficiency.
- Chinese models score around half the success rate of in this test.
- Claude looks weaker on this real-task benchmark, with one case costing nearly 10 times more than similar .