Claude Opus 4.7 looked weaker than 4.6 on an easy-mistake test
ObviousBench is a benchmark made to check whether models make simple, avoidable mistakes when reasoning is low. It looks for failures such as misspelling Google or misunderstanding an everyday situation. Strong model settings are expected to score near the top on this kind of test.
passed 95% on a low setting at a cost of $1.60. Opus 4.6 passed 95% on a high setting at a cost of $0.65. Opus 4.7 only reached 92% even on the highest setting, at a cost of $0.29.
Opus 4.8 passed 95% again on a low setting at a cost of $0.30. The likely issue is that Opus 4.7’s used about one tenth as many as Opus 4.6, then answered too confidently without thinking enough.
Key points
- ObviousBench tests whether models make simple mistakes on easy tasks.
- Opus 4.6 passed 95% on a high setting and cost $0.65.
- Opus 4.7 reached only 92% even on the highest setting and cost $0.29.
- Opus 4.8 passed 95% on a low setting and cost $0.30.
- Opus 4.7 may have used far fewer , leading to overconfident answers.