A die-rolling test shows how LLMs can get stuck on one answer
such as Claude, GPT, and Kimi often answer with “4” when asked to roll a die, instead of spreading answers across 1 to 6. The issue sounds small, but it points to a real RL problem: models may follow familiar answer patterns instead of exploring different choices. A model was so each die number appears roughly one-sixth of the time.
The goal was to make the model follow intended randomness more reliably, rather than collapse into one repeated answer. The work also separates methods that helped from methods that did not.
Key points
- may show a strong bias toward one die result, especially 4.
- The deeper issue is whether a model can explore instead of repeating a learned pattern.
- was used to make numbers 1 through 6 appear at about equal rates.
- The experiment is a toy example for RL exploration problems.
- It is more relevant to agent than to immediate token or cost reduction.