A new test shows coding agents still struggle to build full games
is a benchmark that tests whether can turn a written request into a complete playable game inside a real . It uses 140 Godot tasks across 15 types of games. The test does not only check whether code was written; it also checks whether the game runs, whether a player can interact with it, and whether the screen feedback and presentation make sense.
The best frontier agent reached only 41.46%, and most agents scored below 40%. Agents often manage to create recognizable game rules or movement, but they struggle to finish games with enough content, working , and coherent presentation. The paper provides demos, code, and data, so other models can be tested on the same tasks.
The community discussion focuses on whether medium-size models around 30B to 70B parameters could become strong enough to approach much , especially for coding and writing tasks.
Key points
- contains 140 Godot game-building tasks across 15 game families.
- The benchmark checks runnable gameplay, player interaction, , and presentation.
- The strongest tested frontier agent reached only 41.46%, while most scored below 40%.
- Agents can often build basic mechanics but fail to deliver complete playable games.
- The open code and data can help compare smaller models against larger, more expensive ones.