Qwen3-4B test finds internal entropy is an uneven error signal
A test examined whether Qwen3-4B's internal state could predict wrong answers across about 11,400 examples from seven datasets. The tested measure, , reflects how uncertain or complex the model's internal workspace is while producing an answer.
On fact-retri such as PopQA, combining it with sometimes selected errors more precisely when only a small number of answers could be reviewed. It was particularly helpful for finding questionable answers among those the model presented with high .
It performed much worse than on TruthfulQA, because a wrong belief could still produce a clean, low-entropy internal state. Results also changed sharply by task: a cutoff set using TriviaQA failed on GSM8K, where correct mathematical reasoning naturally had higher entropy, while multiple-choice formatting weakened the signal on CommonSenseQA.
Key points
- The covered about 11,400 examples across seven datasets.
- sometimes improved limited-budget error review for fact retrieval.
- It did not reliably identify confidently held false beliefs on TruthfulQA.
- A cutoff tuned on TriviaQA did not transfer successfully to GSM8K.
- Multiple-choice formatting substantially weakened the signal on CommonSenseQA.