Claude Fable 5 and GPT-5.6 Sol nearly solve puzzle game Baba Is You
Researchers ported levels from the puzzle game Baba Is You into Harbor, a framework for testing AI agents, creating an benchmark called Baba is Harbor. They tested and on Stage 0 (the intro) and Stage 1 (the lake), and both models solved almost every level. Fable 5 was the faster of the two, but still took 4 times longer than a human would.
Although the game has been out for a while, the researchers found no sign that the models had simply memorized solutions from — a contrast with , a widely used coding benchmark where memorization concerns are common. Details on the number of attempts, token usage, and total cost for each model are published in a blog post at quesma.com/blog/baba-is-bench.
Key points
- Baba Is You levels were ported into an agent benchmark called Baba is Harbor
- Both and solved nearly all levels in Stage 0 and Stage 1
- Fable 5 was faster, but still 4x slower than a human
- No evidence of memorized solutions, unlike concerns raised about
- Attempt counts, token usage, and total cost data are published in the blog post